aboutsummaryrefslogtreecommitdiffstats
path: root/router.html
diff options
context:
space:
mode:
Diffstat (limited to 'router.html')
-rw-r--r--router.html151
1 files changed, 136 insertions, 15 deletions
diff --git a/router.html b/router.html
index 5166889..896c4a6 100644
--- a/router.html
+++ b/router.html
@@ -91,7 +91,7 @@
91<li class="toctree-l2"><a class="reference internal" href="receiver.html"><code class="docutils literal"><span class="pre">receiver</span></code> &#8211; Receiver</a></li> 91<li class="toctree-l2"><a class="reference internal" href="receiver.html"><code class="docutils literal"><span class="pre">receiver</span></code> &#8211; Receiver</a></li>
92<li class="toctree-l2 current"><a class="current reference internal" href=""><code class="docutils literal"><span class="pre">router</span></code> &#8211; Router</a></li> 92<li class="toctree-l2 current"><a class="current reference internal" href=""><code class="docutils literal"><span class="pre">router</span></code> &#8211; Router</a></li>
93<li class="toctree-l2"><a class="reference internal" href="sender.html"><code class="docutils literal"><span class="pre">sender</span></code> &#8211; Sender</a></li> 93<li class="toctree-l2"><a class="reference internal" href="sender.html"><code class="docutils literal"><span class="pre">sender</span></code> &#8211; Sender</a></li>
94<li class="toctree-l2"><a class="reference internal" href="utils.html"><code class="docutils literal"><span class="pre">utils</span></code> &#8211; Utilities</a></li> 94<li class="toctree-l2"><a class="reference internal" href="utils/index.html"><code class="docutils literal"><span class="pre">utils</span></code> - Utilities</a></li>
95</ul> 95</ul>
96</li> 96</li>
97<li class="toctree-l1"><a class="reference internal" href="contributing.html">Contributing to EventMQ</a></li> 97<li class="toctree-l1"><a class="reference internal" href="contributing.html">Contributing to EventMQ</a></li>
@@ -149,6 +149,20 @@
149<em class="property">class </em><code class="descclassname">eventmq.router.</code><code class="descname">Router</code><span class="sig-paren">(</span><em>*args</em>, <em>**kwargs</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router" title="Permalink to this definition">¶</a></dt> 149<em class="property">class </em><code class="descclassname">eventmq.router.</code><code class="descname">Router</code><span class="sig-paren">(</span><em>*args</em>, <em>**kwargs</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router" title="Permalink to this definition">¶</a></dt>
150<dd><p>A simple router of messages</p> 150<dd><p>A simple router of messages</p>
151<dl class="method"> 151<dl class="method">
152<dt id="eventmq.router.Router.add_scheduler">
153<code class="descname">add_scheduler</code><span class="sig-paren">(</span><em>scheduler_id</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.add_scheduler" title="Permalink to this definition">¶</a></dt>
154<dd><p>Adds a scheduler to the queue to receive SCHEDULE commands</p>
155<table class="docutils field-list" frame="void" rules="none">
156<col class="field-name" />
157<col class="field-body" />
158<tbody valign="top">
159<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>scheduler_id</strong> (<a class="reference external" href="https://docs.python.org/library/functions.html#str" title="(in Python v2.7)"><em>str</em></a>) &#8211; unique id of the scheduler to add</td>
160</tr>
161</tbody>
162</table>
163</dd></dl>
164
165<dl class="method">
152<dt id="eventmq.router.Router.add_worker"> 166<dt id="eventmq.router.Router.add_worker">
153<code class="descname">add_worker</code><span class="sig-paren">(</span><em>worker_id</em>, <em>queues=None</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.add_worker" title="Permalink to this definition">¶</a></dt> 167<code class="descname">add_worker</code><span class="sig-paren">(</span><em>worker_id</em>, <em>queues=None</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.add_worker" title="Permalink to this definition">¶</a></dt>
154<dd><p>Adds a worker to worker queues</p> 168<dd><p>Adds a worker to worker queues</p>
@@ -157,7 +171,7 @@
157<col class="field-body" /> 171<col class="field-body" />
158<tbody valign="top"> 172<tbody valign="top">
159<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple"> 173<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
160<li><strong>worker_id</strong> &#8211; unique id of the worker to add</li> 174<li><strong>worker_id</strong> (<a class="reference external" href="https://docs.python.org/library/functions.html#str" title="(in Python v2.7)"><em>str</em></a>) &#8211; unique id of the worker to add</li>
161<li><strong>queues</strong> &#8211; queue or queues this worker should be a member of</li> 175<li><strong>queues</strong> &#8211; queue or queues this worker should be a member of</li>
162</ul> 176</ul>
163</td> 177</td>
@@ -167,6 +181,13 @@
167</dd></dl> 181</dd></dl>
168 182
169<dl class="method"> 183<dl class="method">
184<dt id="eventmq.router.Router.clean_up_dead_schedulers">
185<code class="descname">clean_up_dead_schedulers</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.clean_up_dead_schedulers" title="Permalink to this definition">¶</a></dt>
186<dd><p>Loops through the list of schedulers and remove any schedulers who
187the router hasn&#8217;t received a heartbeat in HEARTBEAT_TIMEOUT</p>
188</dd></dl>
189
190<dl class="method">
170<dt id="eventmq.router.Router.clean_up_dead_workers"> 191<dt id="eventmq.router.Router.clean_up_dead_workers">
171<code class="descname">clean_up_dead_workers</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.clean_up_dead_workers" title="Permalink to this definition">¶</a></dt> 192<code class="descname">clean_up_dead_workers</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.clean_up_dead_workers" title="Permalink to this definition">¶</a></dt>
172<dd><p>Loops through the worker queues and removes any workers who haven&#8217;t 193<dd><p>Loops through the worker queues and removes any workers who haven&#8217;t
@@ -174,6 +195,36 @@ responded in HEARTBEAT_TIMEOUT</p>
174</dd></dl> 195</dd></dl>
175 196
176<dl class="method"> 197<dl class="method">
198<dt id="eventmq.router.Router.get_available_worker">
199<code class="descname">get_available_worker</code><span class="sig-paren">(</span><em>queue_name='default'</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.get_available_worker" title="Permalink to this definition">¶</a></dt>
200<dd><p>Gets the job manager with the next available worker for the provided
201queue.</p>
202<table class="docutils field-list" frame="void" rules="none">
203<col class="field-name" />
204<col class="field-body" />
205<tbody valign="top">
206<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><p class="first"><strong>queue_name</strong> (<a class="reference external" href="https://docs.python.org/library/functions.html#str" title="(in Python v2.7)"><em>str</em></a>) &#8211; Name of the queue</p>
207</td>
208</tr>
209<tr class="field-even field"><th class="field-name">Raises:</th><td class="field-body"><ul class="first simple">
210<li><code class="xref py py-exc docutils literal"><span class="pre">NoAvailableWorkerSlotsError</span></code> &#8211; Raised when there are no available</li>
211<li>slots in any the job managers.</li>
212<li><code class="xref py py-exc docutils literal"><span class="pre">UnknownQueueError</span></code> &#8211; Raised when <code class="docutils literal"><span class="pre">queue_name</span></code> is not found in
213self.queues</li>
214</ul>
215</td>
216</tr>
217<tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body"><p class="first">uuid of the job manager with an available worker slot</p>
218</td>
219</tr>
220<tr class="field-even field"><th class="field-name">Return type:</th><td class="field-body"><p class="first last">(str)</p>
221</td>
222</tr>
223</tbody>
224</table>
225</dd></dl>
226
227<dl class="method">
177<dt id="eventmq.router.Router.on_heartbeat"> 228<dt id="eventmq.router.Router.on_heartbeat">
178<code class="descname">on_heartbeat</code><span class="sig-paren">(</span><em>sender</em>, <em>msgid</em>, <em>msg</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.on_heartbeat" title="Permalink to this definition">¶</a></dt> 229<code class="descname">on_heartbeat</code><span class="sig-paren">(</span><em>sender</em>, <em>msgid</em>, <em>msg</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.on_heartbeat" title="Permalink to this definition">¶</a></dt>
179<dd><p>a placeholder for a no-op command. The actual &#8216;logic&#8217; for HEARTBEAT is 230<dd><p>a placeholder for a no-op command. The actual &#8216;logic&#8217; for HEARTBEAT is
@@ -209,11 +260,8 @@ and announces itself.</p>
209 260
210<dl class="method"> 261<dl class="method">
211<dt id="eventmq.router.Router.on_receive_request"> 262<dt id="eventmq.router.Router.on_receive_request">
212<code class="descname">on_receive_request</code><span class="sig-paren">(</span><em>msg</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.on_receive_request" title="Permalink to this definition">¶</a></dt> 263<code class="descname">on_receive_request</code><span class="sig-paren">(</span><em>msg</em>, <em>depth=0</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.on_receive_request" title="Permalink to this definition">¶</a></dt>
213<dd><p>This function is called when a message comes in from the client socket. 264<dd><table class="docutils field-list" frame="void" rules="none">
214It then calls <cite>on_command</cite>. If <cite>on_command</cite> isn&#8217;t found, then a
215warning is created.</p>
216<table class="docutils field-list" frame="void" rules="none">
217<col class="field-name" /> 265<col class="field-name" />
218<col class="field-body" /> 266<col class="field-body" />
219<tbody valign="top"> 267<tbody valign="top">
@@ -224,6 +272,33 @@ warning is created.</p>
224</dd></dl> 272</dd></dl>
225 273
226<dl class="method"> 274<dl class="method">
275<dt id="eventmq.router.Router.on_request">
276<code class="descname">on_request</code><span class="sig-paren">(</span><em>sender</em>, <em>msgid</em>, <em>msg</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.on_request" title="Permalink to this definition">¶</a></dt>
277<dd><p>Process a client REQUEST frame</p>
278</dd></dl>
279
280<dl class="classmethod">
281<dt id="eventmq.router.Router.prioritize_queue_list">
282<em class="property">classmethod </em><code class="descname">prioritize_queue_list</code><span class="sig-paren">(</span><em>unprioritized_iterable</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.prioritize_queue_list" title="Permalink to this definition">¶</a></dt>
283<dd><p>Prioritize a given iterable in the format: ((PRIORITY, OBJ),..)</p>
284<table class="docutils field-list" frame="void" rules="none">
285<col class="field-name" />
286<col class="field-body" />
287<tbody valign="top">
288<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>unprioritized_iterable</strong> (<a class="reference external" href="https://docs.python.org/library/functions.html#iter" title="(in Python v2.7)"><em>iter</em></a>) &#8211; Any list, tuple, etc where the
2890-index key is an integer to use as priority. Largest numbers
290come first.</td>
291</tr>
292<tr class="field-even field"><th class="field-name">Raises:</th><td class="field-body">IndexError - There was no 0-index element.</td>
293</tr>
294<tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body">sorted <code class="xref py py-class docutils literal"><span class="pre">EMQdeque</span></code> with largest priorites being indexed
295smaller. E.g. ((20,&#8217;a&#8217; ), (14, &#8216;b&#8217;), (&#8216;12&#8217;, c))</td>
296</tr>
297</tbody>
298</table>
299</dd></dl>
300
301<dl class="method">
227<dt id="eventmq.router.Router.process_worker_message"> 302<dt id="eventmq.router.Router.process_worker_message">
228<code class="descname">process_worker_message</code><span class="sig-paren">(</span><em>msg</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.process_worker_message" title="Permalink to this definition">¶</a></dt> 303<code class="descname">process_worker_message</code><span class="sig-paren">(</span><em>msg</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.process_worker_message" title="Permalink to this definition">¶</a></dt>
229<dd><p>This method is called when a message comes in from the worker socket. 304<dd><p>This method is called when a message comes in from the worker socket.
@@ -256,14 +331,22 @@ worker slot, so you may see duplicate addresses.</p>
256<p>{&#8216;default&#8217;: [&#8216;w1&#8217;, &#8216;w2&#8217;, &#8216;w1&#8217;, &#8216;w4&#8217;]}</p> 331<p>{&#8216;default&#8217;: [&#8216;w1&#8217;, &#8216;w2&#8217;, &#8216;w1&#8217;, &#8216;w4&#8217;]}</p>
257</dd></dl> 332</dd></dl>
258 333
334<dl class="attribute">
335<dt id="eventmq.router.Router.received_disconnect">
336<code class="descname">received_disconnect</code><em class="property"> = None</em><a class="headerlink" href="#eventmq.router.Router.received_disconnect" title="Permalink to this definition">¶</a></dt>
337<dd><p>Set to True when the router should die.</p>
338</dd></dl>
339
259<dl class="method"> 340<dl class="method">
260<dt id="eventmq.router.Router.requeue_worker"> 341<dt id="eventmq.router.Router.requeue_worker">
261<code class="descname">requeue_worker</code><span class="sig-paren">(</span><em>worker_id</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.requeue_worker" title="Permalink to this definition">¶</a></dt> 342<code class="descname">requeue_worker</code><span class="sig-paren">(</span><em>worker_id</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.requeue_worker" title="Permalink to this definition">¶</a></dt>
262<dd><p>Add a worker back to the pools for which it is a member of.</p> 343<dd><p>Add a worker back to the pools for which it is a member of.</p>
263<div class="admonition note"> 344</dd></dl>
264<p class="first admonition-title">Note</p> 345
265<p class="last">This will (correctly) add duplicate items into the queues.</p> 346<dl class="method">
266</div> 347<dt id="eventmq.router.Router.reset_heartbeat_counters">
348<code class="descname">reset_heartbeat_counters</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.reset_heartbeat_counters" title="Permalink to this definition">¶</a></dt>
349<dd><p>Reset all the counters for heartbeats back to 0</p>
267</dd></dl> 350</dd></dl>
268 351
269<dl class="method"> 352<dl class="method">
@@ -272,6 +355,28 @@ worker slot, so you may see duplicate addresses.</p>
272<dd><p>Kick off router with logging and settings import</p> 355<dd><p>Kick off router with logging and settings import</p>
273</dd></dl> 356</dd></dl>
274 357
358<dl class="attribute">
359<dt id="eventmq.router.Router.scheduler_queue">
360<code class="descname">scheduler_queue</code><em class="property"> = None</em><a class="headerlink" href="#eventmq.router.Router.scheduler_queue" title="Permalink to this definition">¶</a></dt>
361<dd><p><em>Queue for schedulers to use</em></p>
362</dd></dl>
363
364<dl class="attribute">
365<dt id="eventmq.router.Router.schedulers">
366<code class="descname">schedulers</code><em class="property"> = None</em><a class="headerlink" href="#eventmq.router.Router.schedulers" title="Permalink to this definition">¶</a></dt>
367<dd><p>Scheduler clients. Clients are able to send SCHEDULE commands that
368need to be routed to a scheduler, which will keep track of time and
369run the job.
370Contains dictionaries:</p>
371<blockquote>
372<div><dl class="docutils">
373<dt>self.schedulers[&lt;scheduler_zmq_id&gt;] = {</dt>
374<dd>&#8216;hb&#8217;: &lt;last_recv_heartbeat&gt;,</dd>
375</dl>
376<p>}</p>
377</div></blockquote>
378</dd></dl>
379
275<dl class="method"> 380<dl class="method">
276<dt id="eventmq.router.Router.send_ack"> 381<dt id="eventmq.router.Router.send_ack">
277<code class="descname">send_ack</code><span class="sig-paren">(</span><em>socket</em>, <em>recipient</em>, <em>msgid</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.send_ack" title="Permalink to this definition">¶</a></dt> 382<code class="descname">send_ack</code><span class="sig-paren">(</span><em>socket</em>, <em>recipient</em>, <em>msgid</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.send_ack" title="Permalink to this definition">¶</a></dt>
@@ -311,14 +416,27 @@ needed when building messages</p>
311</dd></dl> 416</dd></dl>
312 417
313<dl class="method"> 418<dl class="method">
419<dt id="eventmq.router.Router.send_schedulers_heartbeats">
420<code class="descname">send_schedulers_heartbeats</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.send_schedulers_heartbeats" title="Permalink to this definition">¶</a></dt>
421<dd><p>Send HEARTBEATs to all registered schedulers</p>
422</dd></dl>
423
424<dl class="method">
314<dt id="eventmq.router.Router.send_workers_heartbeats"> 425<dt id="eventmq.router.Router.send_workers_heartbeats">
315<code class="descname">send_workers_heartbeats</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.send_workers_heartbeats" title="Permalink to this definition">¶</a></dt> 426<code class="descname">send_workers_heartbeats</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.send_workers_heartbeats" title="Permalink to this definition">¶</a></dt>
316<dd><p>Send heartbeats to all registered workers.</p> 427<dd><p>Send HEARTBEATs to all registered workers.</p>
428</dd></dl>
429
430<dl class="method">
431<dt id="eventmq.router.Router.sighup_handler">
432<code class="descname">sighup_handler</code><span class="sig-paren">(</span><em>signum</em>, <em>frame</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.sighup_handler" title="Permalink to this definition">¶</a></dt>
433<dd><p>Reloads the configuration and rebinds the ports. Exectued when the
434process receives a SIGHUP from the system.</p>
317</dd></dl> 435</dd></dl>
318 436
319<dl class="method"> 437<dl class="method">
320<dt id="eventmq.router.Router.start"> 438<dt id="eventmq.router.Router.start">
321<code class="descname">start</code><span class="sig-paren">(</span><em>frontend_addr='tcp://127.0.0.1:47290'</em>, <em>backend_addr='tcp://127.0.0.1:47291'</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.start" title="Permalink to this definition">¶</a></dt> 439<code class="descname">start</code><span class="sig-paren">(</span><em>frontend_addr='tcp://127.0.0.1:47291'</em>, <em>backend_addr='tcp://127.0.0.1:47290'</em><span class="sig-paren">)</span><a class="headerlink" href="#eventmq.router.Router.start" title="Permalink to this definition">¶</a></dt>
322<dd><p>Begin listening for connections on the provided connection strings</p> 440<dd><p>Begin listening for connections on the provided connection strings</p>
323<table class="docutils field-list" frame="void" rules="none"> 441<table class="docutils field-list" frame="void" rules="none">
324<col class="field-name" /> 442<col class="field-name" />
@@ -350,8 +468,11 @@ here.</p>
350<dl class="docutils"> 468<dl class="docutils">
351<dt><strong>Keys</strong></dt> 469<dt><strong>Keys</strong></dt>
352<dd><ul class="first last simple"> 470<dd><ul class="first last simple">
353<li>queues: list() of queues the worker belongs to</li> 471<li><code class="docutils literal"><span class="pre">queues</span></code>: list() of queues the worker belongs to</li>
354<li>hb: monotonic timestamp of the last received message from worker</li> 472<li><code class="docutils literal"><span class="pre">hb</span></code>: monotonic timestamp of the last received message from
473worker</li>
474<li><code class="docutils literal"><span class="pre">available_slots</span></code>: int count of jobs this manager can still
475process.</li>
355</ul> 476</ul>
356</dd> 477</dd>
357</dl> 478</dl>