pysap.data#

A module that privides the utility functions to download toy datasets.

get_sample_data(dataset_name, datadir='/home/cg260486/.local/share/pysap', verbose=1)[source]#

Get a sample dataset.

This function download the requested dataset in the ‘$HOME/.local/share/pysap’ directory.

Parameters

  • dataset_name (str) – which sample data you want, must be defined in the ‘SAMPLE_DATA_FILES’ dictionary.

  • verbose (int (optional, default 1)) – control the verbosity level.

Returns

image – the loaded dataset.

Return type

Image

md5_sum_file(fname)[source]#

Calculates the MD5 sum of a file.

Parameters

fname (str (mandatory)) – the path to a file

Returns

md5 – the md5 sum of the input file

Return type

int

progress_bar(ratio, title, bar_length=20, maxsize=40)[source]#

Generate a progress bar

Parameters

  • ratio (float (mandatory)) – the progress status (0<=ratio<1)

  • fname (str (optional)) – the name of the file beeing dowloaded

  • bar_length (int (optional)) – the size of the progress bar

  • maxsize (int (optional)) – use to justify title.

class ResumeURLOpener[source]#

Bases: urllib.request.FancyURLopener

Create sub-class in order to overide error 206. This error means a partial file is being sent, which is fine in this case. Do nothing with this error.

Note: This was adapted from: http://code.activestate.com/recipes/83208-resuming-download-of-a-file/

http_error_206(url, fp, errcode, errmsg, headers, data=None)[source]#
download_file(url, data_dir, resume=True, overwrite=False, verbose=0)[source]#

Load requested file if needed or requested.

Parameters

  • url (str) – the url of the file to be downloaded.

  • data_dir (str) – path of the data directory.

  • resume (bool (optional, default True)) – if True, try to resume partially downloaded files

  • overwrite (bool (optional, default False)) – if True and file already exists, delete it.

  • verbose (int (optional, default 0)) – control the verbosity level.

Returns

absolute path to the downloaded file.

Return type

str

Notes

If, for any reason, the download procedure fails, all downloaded files are removed.

copy_file(path, data_dir, overwrite=False, verbose=0)[source]#

Copy the requested file if needed or requested.

Parameters

  • path (str) – the path to the file to be downloaded.

  • data_dir (str) – path of the data directory.

  • overwrite (bool (optional, default False)) – if True and file already exists, delete it.

  • verbose (int (optional, default 0)) – control the verbosity level.

Returns

copy_file – absolute path to the copied file.

Return type

str