varunayan.processing package

Submodules

• varunayan.processing.data_aggregator module
  • aggregate_by_frequency()
  • aggregate_pressure_levels()
  • set_v_data_agg()
• varunayan.processing.data_filter module
  • filter_netcdf_by_shapefile()
  • get_unique_coordinates_in_polygon()
  • set_v_data_fil()
• varunayan.processing.file_handler module
  • extract_download()
  • find_netcdf_files()
  • set_v_file_han()
• varunayan.processing.variable_lists module

Module contents

varunayan.processing.aggregate_by_frequency(df: DataFrame, frequency: str, keep_original_time: bool = False) → tuple[DataFrame, DataFrame]

Aggregate ERA5 data by the specified frequency for multiple points within a polygon.

The function first aggregates data spatially (across points) for each timestamp, then performs temporal aggregation based on the specified frequency.

Parameters:

• df – DataFrame containing ERA5 data with latitude/longitude points and timestamps

• frequency – One of ‘hourly’, ‘daily’, ‘weekly’, ‘monthly’, ‘yearly’

• keep_original_time – Whether to keep the original valid_time column (default: False)

Returns:

Tuple of (aggregated DataFrame, unique lat/lon DataFrame)

varunayan.processing.aggregate_pressure_levels(df: DataFrame, frequency: str = 'hourly', keep_original_time: bool = False) → tuple[DataFrame, DataFrame]

Aggregate ERA5 pressure level data by the specified frequency.

All variables are aggregated using mean values (both spatially and temporally).

Parameters:

• df – DataFrame containing ERA5 pressure level data with columns: - latitude, longitude: Spatial coordinates - valid_time or time: Timestamps - pressure_level: Pressure level in hPa - Other columns: Meteorological variables

• frequency – One of ‘hourly’, ‘daily’, ‘weekly’, ‘monthly’, ‘yearly’

• keep_original_time – Whether to keep the original time column

Returns:

Tuple of (aggregated DataFrame, unique lat/lon DataFrame)

varunayan.processing.extract_download(zip_or_file_path: str, extract_dir: str | None = None) → List[str]

Extract downloaded file. Handles both single NC file and zip files.

Parameters:

• zip_or_file_path – Path to the downloaded file

• extract_dir – Directory to extract to (optional)

Returns:

List of extracted file paths

varunayan.processing.filter_netcdf_by_shapefile(ds: Dataset, geojson_data: Dict[str, Any]) → DataFrame

Filter a NetCDF dataset to only include grid points that fall within the GeoJSON polygon(s). Internally handles multi-feature GeoJSONs by matching points to each feature individually, then taking the union of all matched points.

Parameters:

• ds – xarray Dataset

• geojson_data – Loaded GeoJSON (as dict or GeoDataFrame)

Returns:

A pandas DataFrame with filtered points.

varunayan.processing.find_netcdf_files(extraction_dir: str) → List[str]

Find all NetCDF files in the extraction directory, including nested directories.

Parameters:

extraction_dir – Directory to search for NetCDF files

Returns:

List of full paths to NetCDF files

varunayan.processing.get_unique_coordinates_in_polygon(ds: Dataset, geojson_data: Dict[str, Any]) → DataFrame

Alternative helper function that returns just the unique lat/lon pairs inside the polygon. This can be useful for other operations or caching coordinate filtering results.

varunayan.processing.set_v_data_agg(verbosity: int)

varunayan.processing.set_v_data_fil(verbosity: int)

varunayan.processing.set_v_file_han(verbosity: int)