Programming, Python

Cool threads

No, not those kinds of threads. Making the right fashion statement is very important, but these are threads of a different cloth.

When you have a task that must be done repeatedly but must wait for external resources to respond, you have a case for threading. Python includes great support for lightweight threads, but there are complications.

Often, parts of what you need to do aren’t thread-safe. They have to run as single instances.

You need to log events, results, output, and anything else you might need to note at run time, but if you have a hundred threads running, chronological logging might be a mess. Log output collated by thread could make life easier.

Exceptions are another conundrum. When a Python thread raises an untrapped exception, the thread terminates. You might have 2,000 things to do, but system resources (or remote resources) to only do ten at a time. Throttling that load requires tracking when threads terminate, whether or not they do so politely.

These and other reasons led me to develop a wrapper around Python’s threading.Thread class.

Overview

The CoolThreads class creates a thread that runs throttling, dispatching of worker threads, logging, command, and control. Any class can be extended through inheritance, but you will rarely need to with CoolThreads. It will perform as constructed for most situations because the target work is performed in worker threads.

CoolThreadOp carries the freight you’re interested in. It is a threaded object you instantiate and hand to CoolThreads for execution.

CoolThreadOp will also rarely need to be inherited and extended, but may not be as much a one-size-fits-all object as the CoolThreads dispatcher. There are use cases for modifying its resources.