{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Comparison of ERA5 and IMD Monthly Rainfall Across Indian States (2000–2024)\n", "\n", "In this notebook, we analyze and compare **monthly precipitation patterns** across Indian states using two key datasets:\n", "- ERA5 (climate reanalysis from ECMWF)\n", "- IMD (Indian Meteorological Department gridded rainfall data)\n", "\n", "We focus on:\n", "- Harmonizing ERA5 and IMD rainfall data at the **state level**\n", "- Evaluating the **correlation between ERA5 and IMD rainfall** for each state\n", "- Visualizing the **spatial pattern of Pearson correlations** to assess agreement between datasets\n", "\n", "We use:\n", "- `varunayan` for extracting ERA5 monthly precipitation (2000–2024)\n", "- `imdlib` for downloading IMD monthly rainfall data\n", "\n", "This study provides insights into how well ERA5 captures regional rainfall variability in India compared to IMD observations, which could be important for validating ERA5 use in climate research and studies for India." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Extract ERA5 Rainfall Data for India\n", "\n", "We use `varunayan` to download **monthly total precipitation (`tp`)** from the ERA5 climate reanalysis dataset for the Indian region from **2000 to 2024**.\n", "\n", "* **North:** 37.5°, **South:** 6°, **East:** 97.5°, **West:** 68°\n", "* **Resolution:** 0.25°\n", "* **Frequency:** Monthly\n", "* **Units:** meters/day\n", "\n", "We will be using the raw data acquired using `varunayan` for this analysis." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\n", "============================================================\u001b[0m\n", "\u001b[0m\u001b[0;34mSTARTING ERA5 SINGLE LEVEL PROCESSING\u001b[0m\u001b[0m\n", "\u001b[0m============================================================\u001b[0m\n", "\u001b[0mRequest ID: rainfall_Indian_region\u001b[0m\n", "\u001b[0mVariables: ['total_precipitation']\u001b[0m\n", "\u001b[0mDate Range: 2000-01-01 to 2024-12-31\u001b[0m\n", "\u001b[0mFrequency: monthly\u001b[0m\n", "\u001b[0mResolution: 0.25°\u001b[0m\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a10143706c794e9e88263f6c41fdbf77", "version_major": 2, "version_minor": 0 }, "text/plain": [ "90d066f69455fe62ad5ba066ba138b9b.zip: 0%| | 0.00/2.45M [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "3827aea08acd4d66ba490e3ca7af259d", "version_major": 2, "version_minor": 0 }, "text/plain": [ "5872f417c2db62948dd5fb8983febe50.zip: 0%| | 0.00/2.48M [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "43fa8e4539ca4cacbf138e6ac4504f3e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "636c8e5597450516c0aae55a252c762e.zip: 0%| | 0.00/2.49M [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\n", "Saving files to output directory: rainfall_Indian_region_output\u001b[0m\n", "\u001b[0m Saved final data to: rainfall_Indian_region_output\\rainfall_Indian_region_monthly_data.csv\u001b[0m\n", "\u001b[0m Saved unique coordinates to: rainfall_Indian_region_output\\rainfall_Indian_region_unique_latlongs.csv\u001b[0m\n", "\u001b[0m Saved raw data to: rainfall_Indian_region_output\\rainfall_Indian_region_raw_data.csv\u001b[0m\n", "\u001b[0m\n", "============================================================\u001b[0m\n", "\u001b[0m\u001b[0;32mPROCESSING COMPLETE\u001b[0m\u001b[0m\n", "\u001b[0m============================================================\u001b[0m\n", "\u001b[0m\n", "\u001b[0;36mRESULTS SUMMARY:\u001b[0m\u001b[0m\n", "\u001b[0m----------------------------------------\u001b[0m\n", "\u001b[0mVariables processed: 1\u001b[0m\n", "\u001b[0mTime period: 2000-01-01 to 2024-12-31\u001b[0m\n", "\u001b[0mFinal output shape: (300, 3)\u001b[0m\n", "\u001b[0mTotal complete processing time: 151.79 seconds\u001b[0m\n", "\u001b[0m\n", "First 5 rows of aggregated data:\u001b[0m\n", "\u001b[0m tp year month\n", "0 0.036963 2000 1\n", "1 0.033683 2000 2\n", "2 0.044679 2000 3\n", "3 0.055403 2000 4\n", "4 0.128061 2000 5\u001b[0m\n", "\u001b[0m\n", "============================================================\u001b[0m\n", "\u001b[0m\u001b[0;34mERA5 SINGLE LEVEL PROCESSING COMPLETED SUCCESSFULLY\u001b[0m\u001b[0m\n", "\u001b[0m============================================================\u001b[0m\n" ] }, { "data": { "text/html": [ "
| \n", " | tp | \n", "year | \n", "month | \n", "
|---|---|---|---|
| 0 | \n", "0.036963 | \n", "2000 | \n", "1 | \n", "
| 1 | \n", "0.033683 | \n", "2000 | \n", "2 | \n", "
| 2 | \n", "0.044679 | \n", "2000 | \n", "3 | \n", "
| 3 | \n", "0.055403 | \n", "2000 | \n", "4 | \n", "
| 4 | \n", "0.128061 | \n", "2000 | \n", "5 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "
| 295 | \n", "0.261888 | \n", "2024 | \n", "8 | \n", "
| 296 | \n", "0.177968 | \n", "2024 | \n", "9 | \n", "
| 297 | \n", "0.133740 | \n", "2024 | \n", "10 | \n", "
| 298 | \n", "0.088298 | \n", "2024 | \n", "11 | \n", "
| 299 | \n", "0.068547 | \n", "2024 | \n", "12 | \n", "
300 rows × 3 columns
\n", "| \n", " | date | \n", "state_name | \n", "monthly_rain_mm | \n", "
|---|---|---|---|
| 0 | \n", "2000-01-01 | \n", "A & N Islands | \n", "38.449201 | \n", "
| 1 | \n", "2000-01-01 | \n", "Andhra Pradesh | \n", "1.987584 | \n", "
| 2 | \n", "2000-01-01 | \n", "Arunachal Pradesh | \n", "147.174445 | \n", "
| 3 | \n", "2000-01-01 | \n", "Assam | \n", "32.347206 | \n", "
| 4 | \n", "2000-01-01 | \n", "Bihar | \n", "1.555414 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "
| 10495 | \n", "2024-12-01 | \n", "Telangana | \n", "19.974696 | \n", "
| 10496 | \n", "2024-12-01 | \n", "Tripura | \n", "0.528455 | \n", "
| 10497 | \n", "2024-12-01 | \n", "Uttar Pradesh | \n", "17.636980 | \n", "
| 10498 | \n", "2024-12-01 | \n", "Uttarakhand | \n", "51.980734 | \n", "
| 10499 | \n", "2024-12-01 | \n", "West Bengal | \n", "4.839537 | \n", "
10500 rows × 3 columns
\n", "| \n", " | time | \n", "lat | \n", "lon | \n", "rain | \n", "
|---|---|---|---|---|
| 0 | \n", "2000-01-01 | \n", "6.5 | \n", "66.50 | \n", "NaN | \n", "
| 1 | \n", "2000-01-01 | \n", "6.5 | \n", "66.75 | \n", "NaN | \n", "
| 2 | \n", "2000-01-01 | \n", "6.5 | \n", "67.00 | \n", "NaN | \n", "
| 3 | \n", "2000-01-01 | \n", "6.5 | \n", "67.25 | \n", "NaN | \n", "
| 4 | \n", "2000-01-01 | \n", "6.5 | \n", "67.50 | \n", "NaN | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 159033775 | \n", "2024-12-31 | \n", "38.5 | \n", "99.00 | \n", "NaN | \n", "
| 159033776 | \n", "2024-12-31 | \n", "38.5 | \n", "99.25 | \n", "NaN | \n", "
| 159033777 | \n", "2024-12-31 | \n", "38.5 | \n", "99.50 | \n", "NaN | \n", "
| 159033778 | \n", "2024-12-31 | \n", "38.5 | \n", "99.75 | \n", "NaN | \n", "
| 159033779 | \n", "2024-12-31 | \n", "38.5 | \n", "100.00 | \n", "NaN | \n", "
159033780 rows × 4 columns
\n", "| \n", " | month | \n", "lat | \n", "lon | \n", "rain | \n", "
|---|---|---|---|---|
| 0 | \n", "2000-01 | \n", "6.5 | \n", "66.50 | \n", "0.0 | \n", "
| 1 | \n", "2000-01 | \n", "6.5 | \n", "66.75 | \n", "0.0 | \n", "
| 2 | \n", "2000-01 | \n", "6.5 | \n", "67.00 | \n", "0.0 | \n", "
| 3 | \n", "2000-01 | \n", "6.5 | \n", "67.25 | \n", "0.0 | \n", "
| 4 | \n", "2000-01 | \n", "6.5 | \n", "67.50 | \n", "0.0 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 5224495 | \n", "2024-12 | \n", "38.5 | \n", "99.00 | \n", "0.0 | \n", "
| 5224496 | \n", "2024-12 | \n", "38.5 | \n", "99.25 | \n", "0.0 | \n", "
| 5224497 | \n", "2024-12 | \n", "38.5 | \n", "99.50 | \n", "0.0 | \n", "
| 5224498 | \n", "2024-12 | \n", "38.5 | \n", "99.75 | \n", "0.0 | \n", "
| 5224499 | \n", "2024-12 | \n", "38.5 | \n", "100.00 | \n", "0.0 | \n", "
5224500 rows × 4 columns
\n", "| \n", " | month | \n", "state_name | \n", "rain | \n", "
|---|---|---|---|
| 0 | \n", "2000-01 | \n", "A & N Islands | \n", "0.000000 | \n", "
| 1 | \n", "2000-01 | \n", "Andhra Pradesh | \n", "0.055613 | \n", "
| 2 | \n", "2000-01 | \n", "Arunachal Pradesh | \n", "35.826280 | \n", "
| 3 | \n", "2000-01 | \n", "Assam | \n", "19.927342 | \n", "
| 4 | \n", "2000-01 | \n", "Bihar | \n", "0.523257 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "
| 10495 | \n", "2024-12 | \n", "Telangana | \n", "17.484431 | \n", "
| 10496 | \n", "2024-12 | \n", "Tripura | \n", "5.540522 | \n", "
| 10497 | \n", "2024-12 | \n", "Uttar Pradesh | \n", "4.986236 | \n", "
| 10498 | \n", "2024-12 | \n", "Uttarakhand | \n", "31.285862 | \n", "
| 10499 | \n", "2024-12 | \n", "West Bengal | \n", "3.514762 | \n", "
10500 rows × 3 columns
\n", "| \n", " | month | \n", "state_name | \n", "rain_imd | \n", "rain_era5 | \n", "
|---|---|---|---|---|
| 0 | \n", "2000-01-01 | \n", "A & N Islands | \n", "0.000000 | \n", "38.449201 | \n", "
| 1 | \n", "2000-01-01 | \n", "Andhra Pradesh | \n", "0.055613 | \n", "1.987584 | \n", "
| 2 | \n", "2000-01-01 | \n", "Arunachal Pradesh | \n", "35.826280 | \n", "147.174445 | \n", "
| 3 | \n", "2000-01-01 | \n", "Assam | \n", "19.927342 | \n", "32.347206 | \n", "
| 4 | \n", "2000-01-01 | \n", "Bihar | \n", "0.523257 | \n", "1.555414 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 10495 | \n", "2024-12-01 | \n", "Telangana | \n", "17.484431 | \n", "19.974696 | \n", "
| 10496 | \n", "2024-12-01 | \n", "Tripura | \n", "5.540522 | \n", "0.528455 | \n", "
| 10497 | \n", "2024-12-01 | \n", "Uttar Pradesh | \n", "4.986236 | \n", "17.636980 | \n", "
| 10498 | \n", "2024-12-01 | \n", "Uttarakhand | \n", "31.285862 | \n", "51.980734 | \n", "
| 10499 | \n", "2024-12-01 | \n", "West Bengal | \n", "3.514762 | \n", "4.839537 | \n", "
10500 rows × 4 columns
\n", "