A Python library for unevenly-spaced time series analysis

Related tags

Deep Learningtraces
Overview

traces

Version PyVersions CircleCI Documentation Status Coverage Status

A Python library for unevenly-spaced time series analysis.

Why?

Taking measurements at irregular intervals is common, but most tools are primarily designed for evenly-spaced measurements. Also, in the real world, time series have missing observations or you may have multiple series with different frequencies: it can be useful to model these as unevenly-spaced.

Traces was designed by the team at Datascope based on several practical applications in different domains, because it turns out unevenly-spaced data is actually pretty great, particularly for sensor data analysis.

Installation

To install traces, run this command in your terminal:

$ pip install traces

Quickstart: using traces

To see a basic use of traces, let's look at these data from a light switch, also known as Big Data from the Internet of Things.

The main object in traces is a TimeSeries, which you create just like a dictionary, adding the five measurements at 6:00am, 7:45:56am, etc.

>>> time_series = traces.TimeSeries()
>>> time_series[datetime(2042, 2, 1,  6,  0,  0)] = 0 #  6:00:00am
>>> time_series[datetime(2042, 2, 1,  7, 45, 56)] = 1 #  7:45:56am
>>> time_series[datetime(2042, 2, 1,  8, 51, 42)] = 0 #  8:51:42am
>>> time_series[datetime(2042, 2, 1, 12,  3, 56)] = 1 # 12:03:56am
>>> time_series[datetime(2042, 2, 1, 12,  7, 13)] = 0 # 12:07:13am

What if you want to know if the light was on at 11am? Unlike a python dictionary, you can look up the value at any time even if it's not one of the measurement times.

>>> time_series[datetime(2042, 2, 1, 11,  0, 0)] # 11:00am
0

The distribution function gives you the fraction of time that the TimeSeries is in each state.

>>> time_series.distribution(
>>>   start=datetime(2042, 2, 1,  6,  0,  0), # 6:00am
>>>   end=datetime(2042, 2, 1,  13,  0,  0)   # 1:00pm
>>> )
Histogram({0: 0.8355952380952381, 1: 0.16440476190476191})

The light was on about 16% of the time between 6am and 1pm.

Adding more data...

Now let's get a little more complicated and look at the sensor readings from forty lights in a house.

How many lights are on throughout the day? The merge function takes the forty individual TimeSeries and efficiently merges them into one TimeSeries where the each value is a list of all lights.

>>> trace_list = [... list of forty traces.TimeSeries ...]
>>> count = traces.TimeSeries.merge(trace_list, operation=sum)

We also applied a sum operation to the list of states to get the TimeSeries of the number of lights that are on.

How many lights are on in the building on average during business hours, from 8am to 6pm?

>>> histogram = count.distribution(
>>>   start=datetime(2042, 2, 1,  8,  0,  0),   # 8:00am
>>>   end=datetime(2042, 2, 1,  12 + 6,  0,  0) # 6:00pm
>>> )
>>> histogram.median()
17

The distribution function returns a Histogram that can be used to get summary metrics such as the mean or quantiles.

It's flexible

The measurements points (keys) in a TimeSeries can be in any units as long as they can be ordered. The values can be anything.

For example, you can use a TimeSeries to keep track the contents of a grocery basket by the number of minutes within a shopping trip.

>>> time_series = traces.TimeSeries()
>>> time_series[1.2] = {'broccoli'}
>>> time_series[1.7] = {'broccoli', 'apple'}
>>> time_series[2.2] = {'apple'}          # puts broccoli back
>>> time_series[3.5] = {'apple', 'beets'} # mmm, beets

To learn more, check the examples and the detailed reference.

More info

Contributing

Contributions are welcome and greatly appreciated! Please visit our guidelines for more info.

Comments
  • Trying to calculate the mean of an empty Histogram fails

    Trying to calculate the mean of an empty Histogram fails

    Running .mean() on an empty Histogram object (Histogram(None, 1000, {0: 0.0})) fails with a divide by zero error:

      File "/src/traces/traces/histogram.py", line 30, in mean
        return weighted_sum / float(self.total())
    ZeroDivisionError: float division by zero
    
    Bug Report 
    opened by vlsd 6
  • How are the plots in the documentation created?

    How are the plots in the documentation created?

    Not a bug, but just curious about how you've plotted the charts in the documentation and what the recommended approach for plotting TimeSeries objects is? I couldn't find a trace of this information in the repo. Thanks in advance!

    opened by Ogaday 5
  • Dev

    Dev

    This covers an initial implementation of the EventSeries features described in #229

    I ended up leaving out the histogram plotting feature as creating reasonable and responsive log binned histograms of time units felt a little outside the scope of this project, though something I may yet tackle.

    Would love a review for readability, test coverage, or feature suggestions!

    opened by nsteins 4
  • add possibility to write ts[start:end] = v to change value on an interval

    add possibility to write ts[start:end] = v to change value on an interval

    I have a use case where I need to change the value of a timeseries on an interval without changing the value outside of the interval, ie do something like ts[start:end] = value. Just setting

    ts[end] = ts[end]   # freezing/anchoring the current value of ts as of [end, ...)
    ts[start] = value      # changing the value as of [start, ...)
    

    may fail as intermediate points in [start,end) may exist ==> we need to remove all intermediate points (which is easy as ts.iterperiods(start,end) provides them nicely).

    I think the function below does it properly (but it would be better integrated in the item to use the slice notation)

    def set_slice(ts, start, end, value):
        """
       ts[start:end] = value ==> call set_slice(ts, start, end, value)
        Set the value of the ts so that
        - on the interval [start, end) we have the new value
        - on [end, ...) we haven't change the value
        - on (..., start) we haven't change the value neither
        We replace the value of the ts on an interval.
    
        :param ts: 
        :param start: 
        :param end: 
        :param value: 
        :return: 
        """
        # for each interval to render
        for i, (s, e, v) in enumerate(list(ts.iterperiods(start, end))):
            # look at all intervals included in the current interval
            # (always at least 1)
            if i == 0:
                # if the first, set initial value to new value of range
                ts[s] = value
            else:
                # otherwise, remove intermediate key
                del ts[s]
        # finish by setting the end of the interval to the previous value
        ts[end] = v
    
    
    
    opened by sdementen 4
  • Values in TimeSeries.distribution() are sentence-cased regardless of how vales were added to the TimeSeries

    Values in TimeSeries.distribution() are sentence-cased regardless of how vales were added to the TimeSeries

    If you are using strings as values in a TimeSeries:

    ts = traces.TimeSeries()
    ts[1] = JUNK
    ts[3] = JANK
    ts[5] = WHAT
    

    If you call something like ts.distrubution(min, max), you would see something like this:

    Histogram(None, 1000, {'Jank': 0.16008504570112725, 'Junk': 0.04229136076598496, 'What': 0.797577092766277})
    

    It looks like somewhere along the line, the string-values are getting sentence-cased. Not sure exactly where yet, but this could be confusing or cause silly bugs if looking-up these objects with the wrong value.

    opened by michaelmoliterno 4
  • Fix conversion of window_size to float breaking timedelta compatiblity

    Fix conversion of window_size to float breaking timedelta compatiblity

    With commit 05a14608d06b06dfc589ae9c247d300b89f956b5, using a timedelta as sampling_period in moving_average throws an exception when converting window_size to a float. Multiplying by 1. (as previously done) serves the same purpose and still allows timedelta to be used.

    opened by cesarrodrig 2
  • Feature Request: linear interpolation for mean

    Feature Request: linear interpolation for mean

    So I recently discovered this nice library and decided to try it since I got unevenly spaced data, however I found out today that the .mean() wasn't doing linear interpolation as I thought it would be:

    >>> from traces import TimeSeries
    >>> t = TimeSeries()
    >>> t[0] = 0
    >>> t[1] = 0
    >>> t[3] = 20
    >>> t.mean(0, 2)
    0.0
    

    With linear interpolation between 2 points we would find that t[2] = 10 and doing the average from 0 to 2 would give us 3.333 in this example. A simple optional argument in mean() to choose the interpolation method would be fantastic, and I really think that it would be useful to many users who are not using traces exclusively for binary data (where linear interpolation would make no sense). I know that we can re-sample the TimeSeries but I think a shortcut like this would be really neat since this library is designed with ease of use in mind.

    Thanks for reading and have a nice day 👋

    opened by Inspirateur 2
  • [Question] How to recreate traces chart?

    [Question] How to recreate traces chart?

    I wonder, how could one plot traces' Signature plot? signature plot ?

    I was wondering if the library has anything to do with the charts (as per the docs that is not the case) but seeing a couple of charts like that in the docs made me think that maybe producing that kind of charts is within the scope of the projects.

    opened by manugarri 2
  • Can't pickle TimeSeries objects

    Can't pickle TimeSeries objects

    [UPDATE] This only seems to happen on python 2.7

    Trying to pickle a TimeSeries object:

    import traces
    ofile = open('test.pkl', 'wb')
    import pickle
    ts = traces.TimeSeries()
    ts[23]="blah"
    ts[2]="foo"
    pickle.dump(ts, ofile)
    

    I get the following error:

    In [9]: pickle.dump(ts, ofile)
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-9-f1eed5bd8d83> in <module>()
    ----> 1 pickle.dump(ts, ofile)
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in dump(obj, file, protocol)
       1374
       1375 def dump(obj, file, protocol=None):
    -> 1376     Pickler(file, protocol).dump(obj)
       1377
       1378 def dumps(obj, protocol=None):
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in dump(self, obj)
        222         if self.proto >= 2:
        223             self.write(PROTO + chr(self.proto))
    --> 224         self.save(obj)
        225         self.write(STOP)
        226
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save(self, obj)
        329
        330         # Save the reduce() output and finally memoize the object
    --> 331         self.save_reduce(obj=obj, *rv)
        332
        333     def persistent_id(self, obj):
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
        423
        424         if state is not None:
    --> 425             save(state)
        426             write(BUILD)
        427
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save(self, obj)
        284         f = self.dispatch.get(t)
        285         if f:
    --> 286             f(self, obj) # Call unbound method with explicit self
        287             return
        288
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save_dict(self, obj)
        653
        654         self.memoize(obj)
    --> 655         self._batch_setitems(obj.iteritems())
        656
        657     dispatch[DictionaryType] = save_dict
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in _batch_setitems(self, items)
        667             for k, v in items:
        668                 save(k)
    --> 669                 save(v)
        670                 write(SETITEM)
        671             return
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save(self, obj)
        284         f = self.dispatch.get(t)
        285         if f:
    --> 286             f(self, obj) # Call unbound method with explicit self
        287             return
        288
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save_dict(self, obj)
        653
        654         self.memoize(obj)
    --> 655         self._batch_setitems(obj.iteritems())
        656
        657     dispatch[DictionaryType] = save_dict
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in _batch_setitems(self, items)
        667             for k, v in items:
        668                 save(k)
    --> 669                 save(v)
        670                 write(SETITEM)
        671             return
    
    /Users/vlad/.pyenv/versions/2.7.13/lib/python2.7/pickle.pyc in save(self, obj)
        304             reduce = getattr(obj, "__reduce_ex__", None)
        305             if reduce:
    --> 306                 rv = reduce(self.proto)
        307             else:
        308                 reduce = getattr(obj, "__reduce__", None)
    
    /Users/vlad/.pyenv/versions/2.7.13/envs/prelude_monitor/lib/python2.7/copy_reg.pyc in _reduce_ex(self, proto)
         68     else:
         69         if base is self.__class__:
    ---> 70             raise TypeError, "can't pickle %s objects" % base.__name__
         71         state = base(self)
         72     args = (self.__class__, base, state)
    
    TypeError: can't pickle instancemethod objects```
    opened by vlsd 2
  • When using a mask with TimeSeries.distribution(), mask.start() is called in `timeseries.py` but `start()` doesn't exist

    When using a mask with TimeSeries.distribution(), mask.start() is called in `timeseries.py` but `start()` doesn't exist

    I think this will be fixed with the next bump; looked for an issue related to this but didn't find one. Feel free to close this out if it was as simple as defining start() for TimeSeries.

    Traceback (most recent call last):
      File "run_plots.py", line 25, in <module>
        make_plots()
      File "/Users/mjfm/projects/modustri/analysis/plots/see_cart_trips.py", line 55, in make_plots
        mask = front_ts,
      File "/Users/mjfm/Virtualenvs/modustri/lib/python2.7/site-packages/traces/timeseries.py", line 622, in distribution
        new_ts = self.slice(mask.start(), mask.end())
    AttributeError: 'TimeSeries' object has no attribute 'start'
    
    opened by michaelmoliterno 2
  • Add `compact` option to `iterperiods()`

    Add `compact` option to `iterperiods()`

    This would merge adjacent periods that have the same value and return them as only one period. Ideally this would be done efficiently, although I'm unclear what that means (store a compact version of the timeseries along with the non-compact one?)

    Enhancement Request 
    opened by vlsd 2
  • `max` for distribution with `start` and `end` gives wrong result

    `max` for distribution with `start` and `end` gives wrong result

    Hello, there seems to be a bug with the Histogram initialization when a start and end are passed.

    versions:

    • python: 3.10.5
    • traces: 0.6.0

    Given the following TimeSeries:

    from traces import TimeSeries
    from pandas import Timestamp
    
    
    ts = TimeSeries(
        {
            Timestamp('2022-10-09 08:48:47'): 5.5,
            Timestamp('2022-10-09 10:36:47'): 51.4,
            Timestamp('2022-10-09 10:38:47'): 15.2,
            Timestamp('2022-10-09 10:38:56'): 0.1,
            Timestamp('2022-10-09 10:41:25'): 4.5
        }
    )
    

    Computing the maximum value with

    ts.distribution().max()
    

    gives 51.4 (as expected)

    However

    ts.distribution(
        start=Timestamp('2022-10-09 07:55:10'),
        end=Timestamp('2022-10-09 10:56:32'),
    ).max()
    

    gives 5.5

    Thank you.

    opened by RuiLoureiro 1
  • No longer maintained?

    No longer maintained?

    This repo looks like it's no longer maintained, with the last PR merged over two years ago. Are you looking for active maintainers? What's the plan for this repo?

    opened by nielsuit227 0
  • Incorrect handling of Numpy array passed as times of measurements

    Incorrect handling of Numpy array passed as times of measurements

    In the following example, although ts1 and ts2 are equal, ts2.distribution() fails with a TypeError as if Numpy arrays weren't recognized properly.
    Somewhat similar to issue #145

    import numpy as np  # Numpy version 1.22.3
    import traces  # traces version 0.6.0
    
    ts1 = traces.TimeSeries(zip(range(4), range(4)), default=0)
    ts2 = traces.TimeSeries(zip(np.arange(4), range(4)), default=0)
    
    ts1 == ts2  # True
    ts1.distribution()  # Histogram({0: 0.3333333333333333, 1: 0.3333333333333333, 2: 0.3333333333333333})
    ts2.distribution()  # TypeError: duration is an unknown type (1)
    
    opened by yportier 0
  • [... list of forty traces.TimeSeries ...] is not functioning.

    [... list of forty traces.TimeSeries ...] is not functioning.

    Hi, Thanks for creating traces. I am trying to learn it. But while I run the following command,

    [... list of forty traces.TimeSeries ...]
    

    I get an error which is mentioned below,

     File "/tmp/ipykernel_51/3316415681.py", line 3
        trace_list = [... list of forty traces.TimeSeries ...]
                             ^
    SyntaxError: invalid syntax
    

    Could anybody please help? Thanks a lot.

    opened by bhavinmoriya 2
  • Allow more flexible type checks on duration

    Allow more flexible type checks on duration

    The current way of checking int and float cannot handle numpy's data types, such as np.int64 and np.float64, which requires extra effort to convert a numpy element into int or float to pass the check.

    Using numeric ABCs solves the problem and allows more flexible "implementations" of integers and real numbers. (Better than the approach in #224, no extra dependencies needed)

    === Updated === Tests can be passed locally. CI seems to complain something about repo_token and marked them as failed.

    opened by zzrcxb 0
Releases(v0.5.1)
prior-based-losses-for-medical-image-segmentation

Repository for papers: Benchmark: Effect of Prior-based Losses on Segmentation Performance: A Benchmark Midl: A Surprisingly Effective Perimeter-based

Rosana EL JURDI 9 Sep 07, 2022
Codebase for arXiv preprint "NeRF++: Analyzing and Improving Neural Radiance Fields"

NeRF++ Codebase for arXiv preprint "NeRF++: Analyzing and Improving Neural Radiance Fields" Work with 360 capture of large-scale unbounded scenes. Sup

Kai Zhang 722 Dec 28, 2022
Scenic: A Jax Library for Computer Vision and Beyond

Scenic Scenic is a codebase with a focus on research around attention-based models for computer vision. Scenic has been successfully used to develop c

Google Research 1.6k Dec 27, 2022
Official codebase for "B-Pref: Benchmarking Preference-BasedReinforcement Learning" contains scripts to reproduce experiments.

B-Pref Official codebase for B-Pref: Benchmarking Preference-BasedReinforcement Learning contains scripts to reproduce experiments. Install conda env

48 Dec 20, 2022
Multi Agent Reinforcement Learning for ROS in 2D Simulation Environments

IROS21 information To test the code and reproduce the experiments, follow the installation steps in Installation.md. Afterwards, follow the steps in E

11 Oct 29, 2022
View model summaries in PyTorch!

torchinfo (formerly torch-summary) Torchinfo provides information complementary to what is provided by print(your_model) in PyTorch, similar to Tensor

Tyler Yep 1.5k Jan 05, 2023
SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data Au

14 Nov 28, 2022
MARS: Learning Modality-Agnostic Representation for Scalable Cross-media Retrieva

Introduction This is the source code of our TCSVT 2021 paper "MARS: Learning Modality-Agnostic Representation for Scalable Cross-media Retrieval". Ple

7 Aug 24, 2022
A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery

PiSL A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery. Sun, F., Liu, Y. and Sun, H., 2021. Physics-informe

Fangzheng (Andy) Sun 8 Jul 13, 2022
Shōgun

The SHOGUN machine learning toolbox Unified and efficient Machine Learning since 1999. Latest release: Cite Shogun: Develop branch build status: Donat

Shōgun ML 2.9k Jan 04, 2023
This repo is to be freely used by ML devs to check the GAN performances without coding from scratch.

GANs for Fun Created because I can! GOAL The goal of this repo is to be freely used by ML devs to check the GAN performances without coding from scrat

Sagnik Roy 13 Jan 26, 2022
Attentive Implicit Representation Networks (AIR-Nets)

Attentive Implicit Representation Networks (AIR-Nets) Preprint | Supplementary | Accepted at the International Conference on 3D Vision (3DV) teaser.mo

29 Dec 07, 2022
Low-dose Digital Mammography with Deep Learning

Impact of loss functions on the performance of a deep neural network designed to restore low-dose digital mammography ====== This repository contains

WANG-AXIS 6 Dec 13, 2022
Code for the paper "Graph Attention Tracking". (CVPR2021)

SiamGAT 1. Environment setup This code has been tested on Ubuntu 16.04, Python 3.5, Pytorch 1.2.0, CUDA 9.0. Please install related libraries before r

122 Dec 24, 2022
Code for paper Adaptively Aligned Image Captioning via Adaptive Attention Time

Adaptively Aligned Image Captioning via Adaptive Attention Time This repository includes the implementation for Adaptively Aligned Image Captioning vi

Lun Huang 45 Aug 27, 2022
Pytorch implementation of the paper "COAD: Contrastive Pre-training with Adversarial Fine-tuning for Zero-shot Expert Linking."

Expert-Linking Pytorch implementation of the paper "COAD: Contrastive Pre-training with Adversarial Fine-tuning for Zero-shot Expert Linking." This is

BoChen 12 Jan 01, 2023
FCN (Fully Convolutional Network) is deep fully convolutional neural network architecture for semantic pixel-wise segmentation

FCN_via_Keras FCN FCN (Fully Convolutional Network) is deep fully convolutional neural network architecture for semantic pixel-wise segmentation. This

Kento Watanabe 48 Aug 30, 2022
DeRF: Decomposed Radiance Fields

DeRF: Decomposed Radiance Fields Daniel Rebain, Wei Jiang, Soroosh Yazdani, Ke Li, Kwang Moo Yi, Andrea Tagliasacchi Links Paper Project Page Abstract

UBC Computer Vision Group 24 Dec 02, 2022
Detectron2 for Document Layout Analysis

Detectron2 trained on PubLayNet dataset This repo contains the training configurations, code and trained models trained on PubLayNet dataset using Det

Himanshu 163 Nov 21, 2022
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

DALL-E in Pytorch Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch. It will also contain CLIP for ranking the ge

Phil Wang 5k Jan 04, 2023