Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 14 additions & 73 deletions 02_activities/assignments/assignment_1.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -26,28 +26,17 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"\n",
"\n",
"from pathlib import Path\n",
"import sys\n",
"\n",
"notebook_dir = Path.cwd()\n",
"src_path = (notebook_dir / \"../../05_src\").resolve()\n",
"\n",
"if str(src_path) not in sys.path:\n",
" sys.path.insert(0, str(src_path)) # insert(0) gives it priority\n",
"\n",
"%load_ext dotenv\n",
"%dotenv \n"
"# Write your code below.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -66,40 +55,17 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from glob import glob\n",
"\n",
"from utils.logger import get_logger\n",
"_logs = get_logger(__name__)\n",
"\n",
"PRICE_DATA = os.getenv(\"PRICE_DATA\")\n",
"parquet_files = glob(os.path.join(PRICE_DATA, \"**/*.parquet\"), recursive = True)\n",
"_logs.info(f'Found {len(parquet_files)} parquet price files.')\n",
"\n",
"\n",
"\n",
" \n",
"# Write your code below.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Please comment:\n",
"\n",
"+ Was it necessary to convert to pandas to calculate the moving average return?\n",
"+ Would it have been better to do it in Dask? Why?\n",
"\n",
"(1 pt)\n",
"\n",
"No it was not necessary. Dask can also calculate it, however it is better to do it in pandas as it stores the data in memory and allows for faster calculation. If it wasn't possible to store in memory then Dask would be a good choice."
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -122,28 +88,11 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"dd_px = dd.read_parquet(parquet_files).set_index(\"ticker\")\n",
"dd_shift = (\n",
" dd_px\n",
" .groupby('ticker', group_keys=False)\n",
" .apply(\n",
" lambda x: x.sort_values('Date', ascending = True)\n",
" .assign(Close_lag_1 = lambda df: df['Close'].shift(1))\n",
" .assign(Adj_Close_lag_1 = lambda df: df['Adj Close'].shift(1)),\n",
" meta = dd_px._meta.assign(Close_lag_1 = 0.0,\n",
" Adj_Close_lag_1 = 0.0))\n",
" )\n",
"\n",
"dd_feat = dd_shift.assign(\n",
" returns = lambda x: x['Close']/x['Close_lag_1'] - 1,\n",
" hi_low_range = lambda x: x['High'] - x['Low']\n",
")\n",
"\n",
"\n",
"# Write your code below.\n",
"\n"
]
},
Expand All @@ -159,18 +108,12 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"df_feat = dd_feat.compute()\n",
"\n",
"\n",
"df_feat[\"returns_10_days\"] = (\n",
" df_feat.groupby(\"ticker\")[\"returns\"]\n",
" .transform(lambda x: x.rolling(10).mean())\n",
")\n",
"df_feat.head(100)"
"# Write your code below.\n",
"\n"
]
},
{
Expand All @@ -182,9 +125,7 @@
"+ Was it necessary to convert to pandas to calculate the moving average return?\n",
"+ Would it have been better to do it in Dask? Why?\n",
"\n",
"(1 pt)\n",
"\n",
"No it was not necessary. Dask can also calculate it, however it is better to do it in pandas as it stores the data in memory and allows for faster calculation. If it wasn't possible to store in memory then Dask would be a good choice."
"(1 pt)"
]
},
{
Expand Down Expand Up @@ -224,7 +165,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "production-env (3.11.14)",
"display_name": "env",
"language": "python",
"name": "python3"
},
Expand All @@ -238,7 +179,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.14"
"version": "3.11.0"
}
},
"nbformat": 4,
Expand Down