=======================
PyNDS: NDS1/NDS2 client
=======================

Python wrapper for nds2-client.  It provides a Python interface for
communicating with NDS1 and NDS2 servers, retrieving lists of available
channels, and fetching channel data.  Retrieve online or offline data from any
of tens of thousands of channels recorded at LIGO gravitational-wave
observatories.


----------
Change Log
----------

2012-11-14 Fixed a bug that prevented fetch() from being able to handle
           minute-trend channels.

           The port argument of the nds.daq constructor is now optional with a
           default value of port=31200.

           Released version 0.7.2.

2012-07-18 Released version 0.7.1.

2012-06-28 Fixed a bug that prevented multiple calls to fetch() or
           request_data() with the same connection object to fail when
           using NDS protocol version 1.

2012-04-14 Bumped version number to 0.7.0 because of Debian packaging error:
           the version 0.5.1 .deb had been mistakenly dubbed 0.6.  Also in this
           release:

           * Removed unmaintained nds_c module.

2012-04-11 Bumped nds2-client dependency from 0.9.0 to 0.9.0c.

2012-04-02 Bumped version to 0.5.2 because of API changes in nds2-client.
           Changed host names in demo scripts to match current server
           configuration at observatories.

2011-07-23 Fixed memory management bug that would cause unexpected behavior if
           an iterator outlived a connection object.
           
           Incremented version number to 0.5.1.

2011-06-21 Bumped version number to 0.5 because of API changes in nds2-client.
           New in pynds version 0.5:
           
           * Wrapped new "static" channel type.
           * Removed wrapping of "rev" field, which was renamed in nds2-client.
		   * The default arguments for daq.request_data are now start=0, stop=0,
             step=1, which sets up online transfer.

2011-05-27 Ported ``nds.fetch`` from Python to C++.  Now the ``nds`` module is
           entirely self contained.
           
           Bumped version number to 0.4.1.

2011-05-24 Added support for complex data type.
           Added ``data_type`` field to ``chan_req`` wrapper.
		   ``nds.fetch`` uses memory more efficiently by preallocating enough
		   space for the entire result.

2011-05-12 Released under terms of GNU General Public License Version 3.

2011-05-03 Bumped version number to 0.4.

2011-05-02 Fixed memory management bug.  The smart_ptr fix didn't cover all
           cases.  It turns out that Boost's ``with_custodian_and_ward`` call
           policy is specifically designed to take care of cases like this where
           a returned value's lifetime depends on the lifetime of ``self``.
           
           Improved documentation of new ``fetch`` routine in README.
           
           Fixed some uninformative exception messages.

2011-04-05 Fixed the channel browser demo, which made use of the ``host`` and
           ``port`` properties, which have been removed.
           
           Resolved an issue where an instance of ``daq_t`` that is pointed to
           by both a Python instance of ``nds.daq`` and an instance of
           ``daq_iterator`` is freed prematurely when the Python instance is
           destroyed.  The solution was to use a ``smart_ptr`` as the
           ``HeldType`` for ``nds.daq``. 

2011-03-08 Added convenience function, ``fetch``, that retrieves data from a given
           start time, a given stop time, and a list of channel names, handling
           all of the details of receiving and concatenating blocks.
           
           Also, the ``daq.request_channel`` method is now overloaded:
           
           * It can take a channel data structure.
           
           * It can take a channel name.  Under the NDS2 protocol, the rate and
             type can be deduced after the first block has been received.
           
           * It can take a channel name, rate, and channel type.

2011-03-07 Bumped nds2-client dependency version to 0.6.1 so that we can remove
           some instructions for a workaround for name collision issue fixed in
           that version.

2011-03-07 Tweaked docstrings in Boost.Python code to enhance readability.
           
           * Turned off C++ signatures.
           
           * Explicitly named the argument ``self`` in all method definitions.
           
           * Edited some docstrings.

2011-03-04 Reworked API a little bit to try out some of Jamie's ideas.
           
           * Moved iterator class into pure C++.
           
           * Moved auxilliary, pure Python methods from Python module to native
             module.
           
           * ``daq.seek`` removed; now ``daq.request_data`` returns iterator.
           
           * Combined overloaded ``recv_channel_list`` into a single function
             with default arguments.
           
           * Removed ``request_channels`` (only singular ``request_channel`` remains).

2011-01-11 Incremented to release 0.3.3.

2010-11-30 Global interpreter lock is now released during data transfers.
           This makes multi-threaded GUI programs feasible.

2010-11-29 Wrapped data_type_t enum to expose channel datatypes.  Now the
           nds.channel class has a data_type field.  Incremented to
           release 0.3.2.

2010-10-14 ``setup.py`` now checks version constraints for ``pkgconfig``
           dependencies.

2010-08-04 Auto-negotiation is now handled by the nds2-client library, instead
           of the Python wrapper.  Incremented to release 0.3.1.

2010-07-08 ``setup.py`` now prints a helpful error message if pkg-config fails.

2010-03-31 Used undocumented behavior of daq_recv_channel_list to query number
           of channels, because on ``ldas-pcdev1.ligo.caltech.edu:31200`` the 
           number of available channels exceeds ``MAX_CHANNELS``.  Bumped
           version number to 0.3.

2010-03-27 Integrated Larne's module with distutils script.  Moved demo scripts
           into dedicated ``demo`` directory.  Moved source files into ``src``.

2010-03-27 Refactored channel browser into a reusable widget.
           Added ``host`` and ``port`` options to channel_browser.py.

2010-03-27 Added PyGTK-based channel browser demo script.

2010-03-26 Added build parameter for non-system installation of Boost.Python.

2010-03-26 Incremented to release 0.2.
           
           * Added ``daq.seek(start, stop, stride)`` command that returns an 
             iterator so that blocks of data can be read in a for loop.
             
             Example::
               
               for data in block.seek(start, stop, stride):
                  plot(data)
           
           * Added ``host`` and ``port`` attributes to ``nds.daq``.
           
           * Added demos.
           
           * Added ``timestamp`` property to ``nds.daq``.
           
2010-03-25 Created version 0.1.


------------
Dependencies
------------

 - Boost.Python (for pynds)
   
   See http://www.boost.org/doc/libs/release/libs/python/doc/
   Available in Debian as the package "libboost-python-dev", or in 
   MacPorts as the package "boost".
   
   To install Boost.Python locally, execute the following commands::
     
     $ wget http://sourceforge.net/projects/boost/files/boost/1.42.0/boost_1_42_0.tar.gz/download
     $ tar xzf boost_1_42_0.tar.gz
     $ pushd boost_1_42_0
     $ ./bootstrap --prefix=/path/to/local
     $ ./bjam --with-python --prefix=/path/to/local install
     $ popd
 
 - nds2-client (for both)
   
   Source download available at:
   https://www.lsc-group.phys.uwm.edu/daswg/wiki/NetworkDataServer2
   
   Make sure you have at least version 0.9.0c, which is here:
   http://www.ligo.caltech.edu/~jzweizig/nds2-release/
   
   nds2-client has some dependencies for the optional Kerberos authentication.
   (To do: find out from J. Zweizig what these dependencies are.)
   If you need to be able to connect to authenticated NDS2 servers, make sure
   that you have these dependencies installed.
   
   To install nds2-client locally, execute the following commands::
     
     $ wget http://ldas-jobs.ligo.caltech.edu/~jzweizig/nds2-release/nds2-client-0.6.1.tar.gz
     $ tar xzf nds2-client-0.6.1.tar.gz
     $ pushd nds2-client-0.6.1
     $ ./configure --prefix=/path/to/local && make install
     $ popd


----------------
Installing pynds
----------------

If you have Boost.Python installed in your compiler's default search path, 
then to install pyNDS, type::
   
   $ python setup.py install

Otherwise, you need to specify where Boost.Python is installed::
   
   $ BOOSTDIR=/path/to/boost  python setup.py install

If you want to install pyNDS in your own home directory, add the --prefix
argument to setup.py::
   
   $ python setup.py install --prefix=/path/to/local


-----
Usage
-----

Examples showing the newly introduced iterator functionality are in 
``demo.py`` and ``demo_online.py`` in the ``demo/`` directory.

Depending on which version of pylab is installed it may be necessary to 
change ``fig.show()`` to ``show()``.  In addition, under OS X it may be neccesary
to add the following before importing ``pylab``:

  from matplotlib import use
  use('MacOSX')

Here is an example Python session using the NDS library::
    
    # Import module
    >>> import nds
    
    # Connect to server
    >>> daq = nds.daq('ldas-pcdev1.ligo.caltech.edu', 31200)
    
    # Get available channels
    >>> channels = daq.recv_channel_list()
    
    # Get all fast channels with names starting with 'H1:LSC-DARM'
    >>> darm_channels = [c for c in channels if c.name.startswith('H1:LSC-DARM') and c.rate == 16384]
    
    # Show name of one of those channels
    >>> darm_channels[10]
    <H1:LSC-DARM_ERR (16384Hz, online)>
    
    # Get 10 seconds of data
    >>> daq.fetch(953537410, 953537410 + 10, 'H1:LSC-DARM_ERR')
    [array([-0.00168277, -0.00168514, -0.00168315, ...,  0.0008413 ,
            0.00083448,  0.0008256 ], dtype=float32)]

It is also possible to retrieve data in small blocks, which may be useful if
you reading online data or if you are building a GUI application.  The
``daq.request_data()`` method returns an iterator; use it as the target of a
``for`` loop::

    # Request 'H1:LSC-DARM_ERR' using a channel object
    >>> daq.request_channel(darm_channels[10])
    
    # Get 10 s of data, in blocks of 1 second
    >>> for block in daq.request_data(953537410, 953537410 + 10, 1):
    ...     # process block
