| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
| |
this makes deployment of the scheduler easier for the devops team
|
| | |
|
| | |
|
| |\
| |
| | |
Scheduler: adds optional support for redis and specifying redis client
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- allow optional support for running the scheduler without a redis
backend for persistent schedule storage. The default is to exit on
startup error. The user can set `redis_startup_error_hard_kill` to
false if they really want to run the scheduler with no backend db.
- allow users the ability to specify a redis client class. this will let
them use redis-cluster if they wish. They can install the appropriate
module and define `redis_client_class` (and if necessary class keyword
arguments via `redis_client_class_kwargs`).
|
| |/
|
|
|
|
|
| |
- Add's support for JSON dictionaries to the config file. This will
allow users to define mappings for certain settings. Initially this
will be used for defining additional class parameters for a custom
Redis class.
|
| |\
| |
| | |
Upgrade to circleci 2.0
|
| | | |
|
| |\ \
| | |
| | | |
Fix bug not allowing env vars to be used without a config file
|
| |/ / |
|
| |\ \
| |/
|/| |
Allow configuration options to be specified as environment variables
|
| |/ |
|
| |\
| |
| | |
Add an option to define max sockets per context
|
| |/ |
|
| |\
| |
| | |
Add job setup and cleanup functions
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds 2 new config options: `job_entry_func` and `job_exit_func`.
These functions are executed before and after every single job
execution. The rationale behind this is before and after each request
Django cleans up stale database connections, so Django jobs need some
way of running this same setup/cleanup functions.
https://github.com/django/django/blob/master/django/db/__init__.py#L57
```
# Register an event to reset transaction state and close connections past
# their lifetime.
def close_old_connections(**kwargs):
for conn in connections.all():
conn.close_if_unusable_or_obsolete()
signals.request_started.connect(close_old_connections)
signals.request_finished.connect(close_old_connections)
```
fixes: #41
|
| | |
|
| |\
| |
| | |
Remove tabs, and add conf file
|
| |/ |
|
| | |
|
| |\
| |
| | |
Add arguments to the scheduler and router
|
| | |\ |
|
| | |/
|/| |
|
| |/ |
|
| |
|
|
| |
- without this the workers get starved.
|
| | |
|
| | |
|
| |\
| |
| | |
Fix bug in router with python2
|
| |/
|
|
|
|
|
|
|
|
|
|
|
| |
- This code seems to have been added awhile ago, but never really
tested in python 2. We shouldn't convert strings to unicode in
python 2 when receiving messages. This causes a TypeError when
forwarding messages:
```
TypeError: Frame 0 (u'719ff6da-ac33-4273-a2bc-6cb9b4...) does not
support the buffer interface.
```
References: 55963796809ce5aacf47c24d8295ab5b1a500ad1
|
| | |
|
| |\
| |
| | |
Fix bug in scheduler; Optimize scheduler code a bit
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
- Fixes a bug when cancelling schedules due to run count. The schedule
cancelled weren't the schedules that were marked for cancellation,
but the schedule that was last checked to see if it needed to be
executed. (use `job_id` from the cancellation loop instead of the job
id from the previous look `k`)
- Consolidate the job cancellation logic. The main loop and
`unschedule` both implemented the same logic. This was consolidated
into a new `cancel_job` method
- Optimize the main loop a bit. If `run_count` is 0 then don't write
to redis and cancel on the next loop. Instead just cancel the
schedule.
|
| |/
|
|
| |
- This should help the response time when trying to debug queue issues.
|
| |\
| |
| | |
mock manager call
|
| | | |
|
| | | |
|
| | | |
|
| |/ |
|
| | |
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
- We saw an issue where sometimes a finish message from the sub
process was being lost so the master process never notified the
router that it was READY for more work. After a few minutes the
router would stop sending jobs to the jobmanager. Changing the
multiprocessing Queue to a mp.Manager proxy queue solved the issue.
- Version bump for 0.3.4.11 to release this fix
|
| | |
| |
| |
| |
| |
| |
| | |
- This was a stab at a fix which didn't fix the issue we were seeing.
It's possible that it caused a memory issue so it is being reverted.
This reverts commit 56cfa885b92b024dc2f70157a0f0b7826ad23e5b.
|
| | |\ |
|
| | |\ \ |
|
| | | | | |
|
| | | | | |
|
| |\ \ \ \
| |_|_|/
|/| | | |
Fix typo in router for metrics gathering
|
| |/ / / |
|