Various logs fixes #231

ael-code · 2015-12-11T09:47:09Z

Various small fixes. Look at the commit history for more details.

Keep in mind that we will move soon to a more robust way for logging configuration

boyska · 2015-12-11T09:53:17Z

At a first glance, everything seems fine. Could you explain the "pin security" thing?

ael-code · 2015-12-11T10:06:43Z

At a first glance, everything seems fine. Could you explain the "pin security" thing?

I don't know what it is :) , it was introduced in the newer version of werkzeug. Since It produces 2 additional lines of code and we are not going to use it, I decided to remove it.

ael-code · 2015-12-17T17:05:22Z

I would like to merge this and release the 0.3 version.

boyska · 2015-12-17T17:28:36Z

I will test NOW

boyska · 2015-12-17T17:33:45Z

It does not work.

TypeError: __init__() got an unexpected keyword argument 'pin_security'

I think this is because you are using a more recent flask version than I am (I'm using Flask==0.10.1). Please resist the urge to force to require a very recent version of flask, or I'll need to create an infinite number of debian packages (see #229 and https://packages.debian.org/jessie/python-flask ).

Apart from this, I can't see what's the improvement of this PR. I don't think this is needed to release 0.3.

ael-code · 2015-12-17T17:36:50Z

Apart from this, I can't see what's the improvement of this PR. I don't think this is needed to release 0.3.

Ok. I've assigned this the next release.

ael-code · 2016-02-28T10:47:59Z

Ok, debugger pin is now documented and seems a nice feature. I've reverted 1fc91bb.

I also modified travis conf to make it run tests with different version of gevent [1.0.1, 1.0.2, 1.1rc5].

boyska · 2016-03-01T20:13:58Z

I am trying to review this PR.

First of all, I'd like a description on the clear advantages of this. I see some refactoring, and they seem good and I am wililng to merge them. But if there is any behavior change I'd like it to be described. I guess that the logging is somewhat "better", but could you be clearer about this?

Secondly, the pin_debugger takes two commits: one does something, the other reverts it. Can you please remove both so that the history is cleaner and shorter?

boyska · 2016-03-03T19:17:11Z

Small note (this is more of a reminder to my self than a comment to the actual code): the way you forced the gevent is not entirely correct. It works, but only because in setup.py we do not specify a version. If we were to specify a version in setup.py (as I am proposing in #254 ) that would not work. The fix is trivial: use before_script instead of before_install, as I did in #256 .

ael-code · 2016-03-04T10:48:16Z

First of all, I'd like a description on the clear advantages of this. I see some refactoring, and they seem good and I am wililng to merge them. But if there is any behavior change I'd like it to be described. I guess that the logging is somewhat "better", but could you be clearer about this?

Now all the output of libreant is handled in the python-logger way this is translated in:

we have now an homogeneous logging
we could control all the loggings from a single place

Logging examples:

Previous logging

2016-03-04 04:44:47,150 [archivant] [DEBUG] initializing with this config: {"ES_HOSTS": null, "ES_INDEXNAME": "libreant", "FSDB_PATH": null}
2016-03-04 04:44:47,151 [archivant] [WARNING] It has not been set any value for FSDB_PATH, file operations will not be supported
2016-03-04 04:44:47,151 [webant] [WARNING] It has not been set any value for 'USERS_DATABASE', all operations about users will be unsupported. Are all admins.
 * Restarting with stat
2016-03-04 04:44:47,513 [archivant] [DEBUG] initializing with this config: {"ES_HOSTS": null, "ES_INDEXNAME": "libreant", "FSDB_PATH": null}
2016-03-04 04:44:47,513 [archivant] [WARNING] It has not been set any value for FSDB_PATH, file operations will not be supported
2016-03-04 04:44:47,514 [webant] [WARNING] It has not been set any value for 'USERS_DATABASE', all operations about users will be unsupported. Are all admins.
 * Debugger is active!
 * Debugger pin code: 103-075-757
Listening on http://0.0.0.0:5000/
2016-03-04 04:44:51,234 [libreantdb.api] [DEBUG] waiting for index "libreant" to be ready
2016-03-04 04:44:51,240 [libreantdb.api] [DEBUG] index "libreant" is now ready
10.10.2.2 - - [2016-03-04 04:44:51] "GET /search?q=%2A%3A%2A HTTP/1.1" 200 11850 0.109837
10.10.2.2 - - [2016-03-04 04:44:51] "GET /static/bootstrap/css/bootstrap.min.css?bootstrap=3.3.5.7 HTTP/1.1" 304 191 0.006109
10.10.2.2 - - [2016-03-04 04:44:51] "GET /static/item_list.css HTTP/1.1" 304 188 0.001848
10.10.2.2 - - [2016-03-04 04:44:51] "GET /static/bootstrap/jquery.min.js?bootstrap=3.3.5.7 HTTP/1.1" 304 190 0.001773
10.10.2.2 - - [2016-03-04 04:44:51] "GET /static/bootstrap/js/bootstrap.min.js?bootstrap=3.3.5.7 HTTP/1.1" 304 190 0.003595
10.10.2.2 - - [2016-03-04 04:44:51] "GET /static/js/ui-main.js HTTP/1.1" 304 187 0.001538

Homogeneous logging

2016-03-04 04:40:58,608 [archivant] [DEBUG] initializing with this config: {"ES_HOSTS": null, "ES_INDEXNAME": "libreant", "FSDB_PATH": null}
2016-03-04 04:40:58,609 [archivant] [WARNING] It has not been set any value for FSDB_PATH, file operations will not be supported
2016-03-04 04:40:58,609 [webant] [WARNING] It has not been set any value for 'USERS_DATABASE', all operations about users will be unsupported. Are all admins.
2016-03-04 04:40:58,630 [werkzeug] [INFO]  * Restarting with stat
2016-03-04 04:40:58,956 [archivant] [DEBUG] initializing with this config: {"ES_HOSTS": null, "ES_INDEXNAME": "libreant", "FSDB_PATH": null}
2016-03-04 04:40:58,957 [archivant] [WARNING] It has not been set any value for FSDB_PATH, file operations will not be supported
2016-03-04 04:40:58,957 [webant] [WARNING] It has not been set any value for 'USERS_DATABASE', all operations about users will be unsupported. Are all admins.
2016-03-04 04:40:58,976 [werkzeug] [WARNING]  * Debugger is active!
2016-03-04 04:40:58,977 [werkzeug] [INFO]  * Debugger pin code: 103-075-757
2016-03-04 04:40:58,991 [webant] [INFO] Listening on http://0.0.0.0:5000/
2016-03-04 04:43:47,381 [libreantdb.api] [DEBUG] waiting for index "libreant" to be ready
2016-03-04 04:43:47,396 [libreantdb.api] [DEBUG] index "libreant" is now ready
2016-03-04 04:43:47,551 [webant] [INFO] 10.10.2.2 - - [2016-03-04 04:43:47] "GET /search?q=%2A%3A%2A HTTP/1.1" 200 11850 0.235244
2016-03-04 04:43:47,576 [webant] [INFO] 10.10.2.2 - - [2016-03-04 04:43:47] "GET /static/item_list.css HTTP/1.1" 304 188 0.004779
2016-03-04 04:43:47,578 [webant] [INFO] 10.10.2.2 - - [2016-03-04 04:43:47] "GET /static/js/ui-main.js HTTP/1.1" 304 187 0.000851

Of course if you don't have gevent >= 1.1 requests logging will be in the old not-homogeneous format. But this is discovered at runtime in a transparent way.

boyska · 2016-03-04T14:46:39Z

ael:

First of all, I'd like a description on the clear advantages of this. I see some refactoring, and they seem good and I am wililng to merge them. But if there is any behavior change I'd like it to be described. I guess that the logging is somewhat "better", but could you be clearer about this?

Now all the output of libreant is handled in the python-logger way this is translated in:

we have now an homogeneous logging

we could control all the loggings from a single place

Now I see. Ok, I will test those things asap so that I can merge
everything.

boyska

run_with_reloader is going to be deprecated from werkzeug/serving

we now test also the brand new gevent-1.1.0 so this commit fixes insomnia-lab#259

ael-code · 2016-03-08T11:57:34Z

Commit history cleaned:

Secondly, the pin_debugger takes two commits: one does something, the other reverts it. Can you please remove both so that the history is cleaner and shorter?

I've removed both the commit related to pin-debugger. If pin-debugger functionality is available well use it

Gevent 1.1.0 release

As stated in #259, gevent 1.1.0 as been released. In order to support the new version I did:

fix gevent supported version in setup.py -> [1.0.1, 1.1.0]
make travis-ci try to install all the stable versions in the supported range: 1.0.1, 1.0.2, 1.1.0

Prayer 🙏

Please consider merging this, it has been submitted on Dec 11, 2015 and makes our logs so sexy.

boyska · 2016-03-08T20:33:42Z

ael:

Commit history cleaned:

Secondly, the pin_debugger takes two commits: one does something, the other reverts it. Can you please remove both so that the history is cleaner and shorter?

I've removed both the commit related to pin-debugger. If pin-debugger functionality is available well use it

Gevent 1.1.0 release

As stated in #259, gevent 1.1.0 as been released. In order to support the new version I did:

fix gevent supported version in setup.py -> [1.0.1, 1.1.0]

make travis-ci try to install all the stable versions in the supported range: 1.0.1, 1.0.2, 1.1.0

But is this useful at all? What I mean is that the code related to
gevent is never run during tests, so I don't think it will be useful to
do all the unit tests with different gevent version

Prayer 🙏

Please consider merging this, it has been submitted on Dec 11, 2015 and makes our logs so sexy.

ok

Various logs improvements and fixes

ael-code · 2016-03-08T23:10:16Z

But is this useful at all? What I mean is that the code related to gevent is never run during tests, so I don't think it will be useful to do all the unit tests with different gevent version

Yeap I know. I hope we'll be able to test it in the near future

ael-code added the enhancement label Dec 11, 2015

ael-code added this to the 0.4 milestone Dec 17, 2015

ael-code force-pushed the log_stuff branch 2 times, most recently from cb777ca to 7e4e04a Compare February 28, 2016 10:41

ael-code added system integration tests labels Feb 28, 2016

ael-code modified the milestones: 0.3.1, 0.4 Mar 1, 2016

boyska mentioned this pull request Mar 1, 2016

better library versions specification #254

Open

ael-code mentioned this pull request Mar 6, 2016

dependencies: gevent 1.1.0 has been released #259

Closed

ael-code added 7 commits March 8, 2016 11:47

import werkzeug reloader from the original source

20890d2

run_with_reloader is going to be deprecated from werkzeug/serving

changed default log level to INFO

9a74cdd

added werkzeug to handled loggers

5282870

use logger for printing address:port

9f60475

Use custom logger on gevent>=1.1

1ba0d9a

fix gevent supported version to [1.0.1, 1.1.0] on setup.py

286a3ec

CI: Test against different gevent versions

9eb35aa

we now test also the brand new gevent-1.1.0 so this commit fixes insomnia-lab#259

ael-code force-pushed the log_stuff branch from cbda847 to 9eb35aa Compare March 8, 2016 11:41

boyska added a commit that referenced this pull request Mar 8, 2016

Merge pull request #231 from ael-code/log_stuff

0981013

Various logs improvements and fixes

boyska merged commit 0981013 into insomnia-lab:dev Mar 8, 2016

ael-code deleted the log_stuff branch March 18, 2016 16:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Various logs fixes #231

Various logs fixes #231

ael-code commented Dec 11, 2015

boyska commented Dec 11, 2015

ael-code commented Dec 11, 2015

ael-code commented Dec 17, 2015

boyska commented Dec 17, 2015

boyska commented Dec 17, 2015

ael-code commented Dec 17, 2015

ael-code commented Feb 28, 2016

boyska commented Mar 1, 2016

boyska commented Mar 3, 2016

ael-code commented Mar 4, 2016

boyska commented Mar 4, 2016

ael-code commented Mar 8, 2016

boyska commented Mar 8, 2016

Commit history cleaned:

Gevent 1.1.0 release

Prayer 🙏

ael-code commented Mar 8, 2016

Various logs fixes #231

Various logs fixes #231

Conversation

ael-code commented Dec 11, 2015

boyska commented Dec 11, 2015

ael-code commented Dec 11, 2015

ael-code commented Dec 17, 2015

boyska commented Dec 17, 2015

boyska commented Dec 17, 2015

ael-code commented Dec 17, 2015

ael-code commented Feb 28, 2016

boyska commented Mar 1, 2016

boyska commented Mar 3, 2016

ael-code commented Mar 4, 2016

Logging examples:

Previous logging

Homogeneous logging

boyska commented Mar 4, 2016

ael-code commented Mar 8, 2016

Commit history cleaned:

Gevent 1.1.0 release

Prayer 🙏

boyska commented Mar 8, 2016

Commit history cleaned:

Gevent 1.1.0 release

Prayer 🙏

ael-code commented Mar 8, 2016