collect_csv_files

process_ais_data.collect_csv_files(options, directories, meta_params)

Traverses the directory containing the decompressed AIS data to get the csv names for further processing.

Uses the os library to find all csv files within the directory defined by directories, populating the csv_files list for later reading of all valid csv files found and logging file metadata in all_files_meta.

Parameters

  • options (dict) – The script options specified in the config_file.

  • directories (dict) – The input and output paths and files specified in the config_file.

  • meta_params (dict) – The time and zone boundaries specified in the config_file.

Returns

A list with paths to all valid csv files found and a dictionary with the year, month, and zone corresponding to each csv file’s origin.

Return type

tuple