traffic.core.lazy

class traffic.core.lazy.LazyTraffic(wrapped_t, stacked_ops, iterate_kw=None, tqdm_kw=None)

Bases: object

In the following example, lazy_t is not evaluated:

>>> lazy_t = t.filter().resample('10s')
>>> type(t_lazy)
traffic.core.lazy.LazyTraffic

You need to call the .eval() method for that.

eval(max_workers=1, desc=None, cache_file=None)

The result can only be accessed after a call to eval().

Parameters:

  • max_workers (int) – (default: 1) Multiprocessing is usually worth it. However, a sequential processing is triggered by default. Keep the value close to the number of cores of your processor. If memory becomes a problem, stick to the default.

  • desc (None | str) – (default: None) If not None, a tqdm progressbar is displayed with this parameter.

  • cache_file (str | Path | None) – (default: None) If not None, store the results in cache_file and load the results from the file if it exists.

Return type:

None | ‘Traffic

Example usage:

The following call

>>> t_lazy.eval(max_workers=4, desc="preprocessing")

is equivalent to the multiprocessed version of

>>> Traffic.from_flights(
...     flight.filter().resample("10s")
...     for flight in tqdm(t, desc="preprocessing")
... )

When many operations are stacked, this call is more efficient, esp. on large structures, than as many full iterations on the Traffic structure.

Backward compatibility is ensured by an automatic call to eval() with default options.

>>> t_lazy.to_pickle("output_file.pkl")
WARNING:root:.eval() has been automatically appended for you.
Check the documentation for more options.
traffic.core.lazy.apply(stacked_ops, idx, flight)

Recursively applies all operations on each Flight. :rtype: Optional[Flight]

  • map operations return Flight

  • filter operations return a Flight if True, otherwise None

Note that the only valid reduce operation is Traffic.from_flights.

traffic.core.lazy.lazy_evaluation(default=False, idx_name=None)

A decorator to delegate methods to Flight in a lazy manner.

Each decorated Traffic method returns a LazyTraffic structure with the corresponding operation stacked.

When the default option is set to True, the method returns a Traffic when called on Traffic but is stacked before applications on Flight objects if called on a LazyTraffic.

Parameters:

  • default (None | bool) – (default: False) If set to True, the Traffic implementation is used when called on a Traffic structure; and the Flight implementation is stacked when called on a LazyTraffic structure.

  • idx_name (None | str) – (default: None) If the method needs the index (from enumerate) produced during the iteration, specify the name of the corresponding argument. (see {Traffic, Flight}.assign_id for an example)

Return type:

Callable[[Callable[[Concatenate[Traffic, ParamSpec(P)]], TypeVar(R)]], Callable[[Concatenate[Traffic, ParamSpec(P)]], Union[TypeVar(R), LazyTraffic]]]