{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Comparison of ERA5 and IMD Daily Temperature Extremes Across Maharashtra (2024)\n", "\n", "In this notebook, we analyze and compare **daily temperature extremes patterns** across Maharashtra using two key datasets:\n", "- ERA5: Climate reanalysis data from ECMWF\n", "- IMD: Gridded daily temperature extremes from the Indian Meteorological Department\n", "\n", "We focus on:\n", "- Harmonizing ERA5 and IMD temperature data\n", "- Evaluating the **correlation between ERA5 and IMD temperature extremes** at each grid point\n", "- Visualizing the **spatial pattern of Pearson correlations** to assess agreement between datasets\n", "\n", "We use:\n", "- `varunayan` for extracting ERA5 hourly minimum and maximum temperatures\n", "- `imdlib` for accessing IMD daily tmin and tmax data\n", "\n", "This study provides insights into how well ERA5 captures regional temperature extremes in Maharashtra compared to IMD observations, which could be important for validating ERA5 use in climate research and studies for Maharashtra." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Step 1: Extract ERA5 Temperature Extremes Data for Maharashtra\n", "\n", "We use `varunayan` to download **hourly minimum and maximum temperature (`mn2t` and `mx2t`)** from the ERA5 climate reanalysis dataset for the Maharashtra region, covering the year **2024**.\n", "\n", "* **North:** 21.5°, **South:** 15.5°, **East:** 80.5°, **West:** 72.5°\n", "* **Resolution:** 1°\n", "* **Frequency:** hourly\n", "* **Units:** Kelvin\n", "\n", "We will be using the raw data acquired using `varunayan` for this analysis." ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\n", "============================================================\u001b[0m\n", "\u001b[0m\u001b[0;34mSTARTING ERA5 SINGLE LEVEL PROCESSING\u001b[0m\u001b[0m\n", "\u001b[0m============================================================\u001b[0m\n", "\u001b[0mRequest ID: min_max_temp_maha_2020\u001b[0m\n", "\u001b[0mVariables: ['maximum_2m_temperature_since_previous_post_processing', 'minimum_2m_temperature_since_previous_post_processing']\u001b[0m\n", "\u001b[0mDate Range: 2020-01-01 to 2020-12-31\u001b[0m\n", "\u001b[0mFrequency: hourly\u001b[0m\n", "\u001b[0mResolution: 1°\u001b[0m\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "eb1bcf7b25c84523b181d0169e4d56c8", "version_major": 2, "version_minor": 0 }, "text/plain": [ "f4647b5e9c07c26ec2076441dc992a97.zip: 0%| | 0.00/139k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "aab29f4488ba49c99d35d4c54f44e1cf", "version_major": 2, "version_minor": 0 }, "text/plain": [ "488b005a015b26ac7e19fb6ddad74735.zip: 0%| | 0.00/139k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "2d2e968505f9462692325163b124431e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "71fdfa0b5d901da8541a659594861dfc.zip: 0%| | 0.00/224k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e1f1c9f7e4824d878352d5c0461ee2b8", "version_major": 2, "version_minor": 0 }, "text/plain": [ "fe94d391e092f11bbbd4231c8ccabc78.zip: 0%| | 0.00/139k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d892c61410134ac98a4597c87d6c2f29", "version_major": 2, "version_minor": 0 }, "text/plain": [ "abd170e286271e9f8cca9d2f9e738568.zip: 0%| | 0.00/237k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "dd78714a3b7f453fb8f6ad20b6fa56d4", "version_major": 2, "version_minor": 0 }, "text/plain": [ "d9e5ad79a2bc54cd5ac7e0e0f0d5084b.zip: 0%| | 0.00/140k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a37fb632776442a989482b58b59ef207", "version_major": 2, "version_minor": 0 }, "text/plain": [ "e42791d0381a75060630a057686131bb.zip: 0%| | 0.00/232k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "17b441d3b8c546d5a7b52e3de74184e3", "version_major": 2, "version_minor": 0 }, "text/plain": [ "8bfdce668e6bd36d3589f9574c2ef681.zip: 0%| | 0.00/140k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "cb0ff1aa81e747a2b5db18bf1c50baeb", "version_major": 2, "version_minor": 0 }, "text/plain": [ "b21b3aee066dd0920fb358712d67726e.zip: 0%| | 0.00/237k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e94bc23c582f495899e52cd5eafe3e1d", "version_major": 2, "version_minor": 0 }, "text/plain": [ "6a7126571da9b52e342eb7022fd05116.zip: 0%| | 0.00/140k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "5cd4e878818e43cf8f9f2a8de0ba0cb8", "version_major": 2, "version_minor": 0 }, "text/plain": [ "b0e9cc0aefc668f78a0c060fe9b057fd.zip: 0%| | 0.00/230k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a39febc54fa747faa32518e23e341a6f", "version_major": 2, "version_minor": 0 }, "text/plain": [ "268ad8dc1b6fba4812da12b91db502b4.zip: 0%| | 0.00/138k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "926b51327b924702aeff452dc8426765", "version_major": 2, "version_minor": 0 }, "text/plain": [ "61f4992b19f59185cfae8f944c2250c3.zip: 0%| | 0.00/138k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "18f450639a574805bb3172785c94b6dc", "version_major": 2, "version_minor": 0 }, "text/plain": [ "60cf82c93e5efa9432dfe37679ba023.zip: 0%| | 0.00/137k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9d69d250b4cb41dead08002354ac52ca", "version_major": 2, "version_minor": 0 }, "text/plain": [ "b2194964cfdcee59f625a100dca44e54.zip: 0%| | 0.00/137k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "89d52fe97b9745549695e994142e10a3", "version_major": 2, "version_minor": 0 }, "text/plain": [ "fdcef0737cd44fc74523fa04b1b00514.zip: 0%| | 0.00/232k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "58c8ece39ea84d6ea8af80056519084d", "version_major": 2, "version_minor": 0 }, "text/plain": [ "3bfab345b02d006ccef405620a032f40.zip: 0%| | 0.00/136k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "1ed48dfe4a584f45abf5d396b2016f79", "version_major": 2, "version_minor": 0 }, "text/plain": [ "981f694dcc7a486c0486d3f2facc9210.zip: 0%| | 0.00/228k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "df91bbdcae9041cc8ce9f2a08e91994c", "version_major": 2, "version_minor": 0 }, "text/plain": [ "edfb093789dc1614a6aa8dc7d3c81d6e.zip: 0%| | 0.00/137k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0fba9c5c29ff46b6884b706aa328887a", "version_major": 2, "version_minor": 0 }, "text/plain": [ "bd1d9e16d1e9e6381780f53911d43fd3.zip: 0%| | 0.00/234k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "3de70c6360114469997eab3cc71d11d8", "version_major": 2, "version_minor": 0 }, "text/plain": [ "38e064ee4e7fefbdaad3dc686ff7cf4f.zip: 0%| | 0.00/137k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "f60eb0fb6d194aec8d753a3928d0a54e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "60631bbf43933699a38583eea643f5ba.zip: 0%| | 0.00/230k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "78f5aec527bb4e789eb871ded67ce72a", "version_major": 2, "version_minor": 0 }, "text/plain": [ "a9211b816799b2fd3f6e8e1e71c12d2d.zip: 0%| | 0.00/139k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9aa22f34225e4c9990543035688f1d9e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "bbe49b934ba8da1da7a5fa4e67261754.zip: 0%| | 0.00/236k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "7190d3d75ba04a7b92088b33752963fa", "version_major": 2, "version_minor": 0 }, "text/plain": [ "2b226c87d3eaa429e71b65b26b780b75.zip: 0%| | 0.00/139k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "f125ce8795304219aeb72f22b7c75f28", "version_major": 2, "version_minor": 0 }, "text/plain": [ "fc56caf67165a27a337901b5fe4f644f.zip: 0%| | 0.00/139k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "4986ca44026a4aaf984d0314ba9383da", "version_major": 2, "version_minor": 0 }, "text/plain": [ "1ae6a7b73546faa8dbcdb7c84279c14a.zip: 0%| | 0.00/45.6k [00:00, ?B/s]" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\u001b[0m\n", "Saving files to output directory: min_max_temp_maha_2020_output\u001b[0m\n", "\u001b[0m Saved final data to: min_max_temp_maha_2020_output\\min_max_temp_maha_2020_hourly_data.csv\u001b[0m\n", "\u001b[0m Saved unique coordinates to: min_max_temp_maha_2020_output\\min_max_temp_maha_2020_unique_latlongs.csv\u001b[0m\n", "\u001b[0m Saved raw data to: min_max_temp_maha_2020_output\\min_max_temp_maha_2020_raw_data.csv\u001b[0m\n", "\u001b[0m\n", "============================================================\u001b[0m\n", "\u001b[0m\u001b[0;32mPROCESSING COMPLETE\u001b[0m\u001b[0m\n", "\u001b[0m============================================================\u001b[0m\n", "\u001b[0m\n", "\u001b[0;36mRESULTS SUMMARY:\u001b[0m\u001b[0m\n", "\u001b[0m----------------------------------------\u001b[0m\n", "\u001b[0mVariables processed: 2\u001b[0m\n", "\u001b[0mTime period: 2020-01-01 to 2020-12-31\u001b[0m\n", "\u001b[0mFinal output shape: (8784, 7)\u001b[0m\n", "\u001b[0mTotal complete processing time: 2865.08 seconds\u001b[0m\n", "\u001b[0m\n", "First 5 rows of aggregated data:\u001b[0m\n", "\u001b[0m mx2t mn2t date year month day hour\n", "0 299.386475 286.384277 2020-01-01 2020 1 1 0\n", "1 299.424072 286.346191 2020-01-01 2020 1 1 1\n", "2 299.484619 286.520264 2020-01-01 2020 1 1 2\n", "3 299.594971 286.739990 2020-01-01 2020 1 1 3\n", "4 299.720703 288.402100 2020-01-01 2020 1 1 4\u001b[0m\n", "\u001b[0m\n", "============================================================\u001b[0m\n", "\u001b[0m\u001b[0;34mERA5 SINGLE LEVEL PROCESSING COMPLETED SUCCESSFULLY\u001b[0m\u001b[0m\n", "\u001b[0m============================================================\u001b[0m\n" ] }, { "data": { "text/html": [ "
| \n", " | mx2t | \n", "mn2t | \n", "date | \n", "year | \n", "month | \n", "day | \n", "hour | \n", "
|---|---|---|---|---|---|---|---|
| 0 | \n", "299.386475 | \n", "286.384277 | \n", "2020-01-01 | \n", "2020 | \n", "1 | \n", "1 | \n", "0 | \n", "
| 1 | \n", "299.424072 | \n", "286.346191 | \n", "2020-01-01 | \n", "2020 | \n", "1 | \n", "1 | \n", "1 | \n", "
| 2 | \n", "299.484619 | \n", "286.520264 | \n", "2020-01-01 | \n", "2020 | \n", "1 | \n", "1 | \n", "2 | \n", "
| 3 | \n", "299.594971 | \n", "286.739990 | \n", "2020-01-01 | \n", "2020 | \n", "1 | \n", "1 | \n", "3 | \n", "
| 4 | \n", "299.720703 | \n", "288.402100 | \n", "2020-01-01 | \n", "2020 | \n", "1 | \n", "1 | \n", "4 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 8779 | \n", "301.058594 | \n", "288.187988 | \n", "2020-12-31 | \n", "2020 | \n", "12 | \n", "31 | \n", "19 | \n", "
| 8780 | \n", "301.026367 | \n", "287.927979 | \n", "2020-12-31 | \n", "2020 | \n", "12 | \n", "31 | \n", "20 | \n", "
| 8781 | \n", "300.906494 | \n", "287.406494 | \n", "2020-12-31 | \n", "2020 | \n", "12 | \n", "31 | \n", "21 | \n", "
| 8782 | \n", "300.800781 | \n", "286.872314 | \n", "2020-12-31 | \n", "2020 | \n", "12 | \n", "31 | \n", "22 | \n", "
| 8783 | \n", "300.804199 | \n", "286.522705 | \n", "2020-12-31 | \n", "2020 | \n", "12 | \n", "31 | \n", "23 | \n", "
8784 rows × 7 columns
\n", "| \n", " | time | \n", "lat | \n", "lon | \n", "tmax | \n", "
|---|---|---|---|---|
| 0 | \n", "2020-01-01 | \n", "15.5 | \n", "72.5 | \n", "99.900002 | \n", "
| 1 | \n", "2020-01-01 | \n", "15.5 | \n", "73.5 | \n", "31.408098 | \n", "
| 2 | \n", "2020-01-01 | \n", "15.5 | \n", "74.5 | \n", "31.509253 | \n", "
| 3 | \n", "2020-01-01 | \n", "15.5 | \n", "75.5 | \n", "31.085457 | \n", "
| 4 | \n", "2020-01-01 | \n", "15.5 | \n", "76.5 | \n", "30.606255 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 23053 | \n", "2020-12-31 | \n", "21.5 | \n", "76.5 | \n", "28.515602 | \n", "
| 23054 | \n", "2020-12-31 | \n", "21.5 | \n", "77.5 | \n", "27.787882 | \n", "
| 23055 | \n", "2020-12-31 | \n", "21.5 | \n", "78.5 | \n", "26.746124 | \n", "
| 23056 | \n", "2020-12-31 | \n", "21.5 | \n", "79.5 | \n", "26.715961 | \n", "
| 23057 | \n", "2020-12-31 | \n", "21.5 | \n", "80.5 | \n", "27.381311 | \n", "
23058 rows × 4 columns
\n", "| \n", " | time | \n", "lat | \n", "lon | \n", "tmin | \n", "
|---|---|---|---|---|
| 0 | \n", "2020-01-01 | \n", "15.5 | \n", "72.5 | \n", "99.900002 | \n", "
| 1 | \n", "2020-01-01 | \n", "15.5 | \n", "73.5 | \n", "17.982029 | \n", "
| 2 | \n", "2020-01-01 | \n", "15.5 | \n", "74.5 | \n", "18.658150 | \n", "
| 3 | \n", "2020-01-01 | \n", "15.5 | \n", "75.5 | \n", "17.838404 | \n", "
| 4 | \n", "2020-01-01 | \n", "15.5 | \n", "76.5 | \n", "18.578913 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 23053 | \n", "2020-12-31 | \n", "21.5 | \n", "76.5 | \n", "12.886724 | \n", "
| 23054 | \n", "2020-12-31 | \n", "21.5 | \n", "77.5 | \n", "12.003990 | \n", "
| 23055 | \n", "2020-12-31 | \n", "21.5 | \n", "78.5 | \n", "11.423929 | \n", "
| 23056 | \n", "2020-12-31 | \n", "21.5 | \n", "79.5 | \n", "10.512906 | \n", "
| 23057 | \n", "2020-12-31 | \n", "21.5 | \n", "80.5 | \n", "11.225183 | \n", "
23058 rows × 4 columns
\n", "