-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathrequirements.txt
More file actions
265 lines (199 loc) · 9.54 KB
/
Copy pathrequirements.txt
File metadata and controls
265 lines (199 loc) · 9.54 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
# requirements.txt - list all the packages the project needs
#
# In this file, ignore hashes (#) - they are just used to create comments.
# Lines starting with a hash are ignored when installing packages using this file.
# ======================================================
# IMPORTANT: The contents of this file varies by project
# ======================================================
# Some common dependencies are provided in this example.
# Comment them in or out as you need them.
# ======================================================
# STEP A - CREATE A LOCAL PROJECT VIRTUAL ENV (.venv)
# ======================================================
# This option uses the most current or default Python -
# if an older version is required, use the ADVANCED OPTION below.
# Create your local project virtual environment
# This step ensures you have an isolated Python environment for your project.
# This is typically just done once at the beginning of a project.
# If it gets messed up, we can delete .venv and recreate it at any time.
# Run the following command to create a virtual environment in the project root.
### On Windows, Use PowerShell (not cmd) - don't include the #:
# py -m venv .venv
### On Mac/Linux, Use zsh or bash (or PowerShell) - don't include the #:
# python3 -m venv .venv
### If VS Code asks: We noticed a new environment has been created.
# Do you want to select it for the workspace folder?
# Click Yes.
# ======================================================
# STEP A (ADVANCED OPTION) - ONLY WHEN OLDER PYTHON VERSION IS REQUIRED
# ======================================================
### IMPORTANT:
### If the project requires a large tool like Apache Kafka,
### we may need to install an earlier version of Python
### and specify the required version when we create the virtual environment.
### On Windows, Use PowerShell (not cmd) - don't include the #:
# py -3.11 -m venv .venv
### On Mac - don't include the #:
# brew install python@3.11
# python3 -m venv .venv
### On Linux - don't include the #:
# sudo apt update
# sudo apt install python3.11 python3.11-venv
# python3.11 -m venv .venv
# ======================================================
# STEP B - ALWAYS ACTIVATE THE (.venv) WHEN OPENING A NEW TERMINAL
# ======================================================
# ALWAYS activate the .venv before working on the project.
# Activate whenever you open a new terminal.
### Windows PowerShell Command (don't include the #):
# .\.venv\Scripts\activate
### Mac/Linux Command (don't include the #):
# source .venv/bin/activate
# Verify: When active, you can usually see (.venv) in the terminal.
# If using a Jupyter notebook, select the kernel associated with your project (.venv).
# ======================================================
# STEP C - INSTALL PACKAGES INTO (.venv) AS NEEDED
# ======================================================
# Install necessary packages listed below with this command:
# Keep packages updated with the most recent versions.
# When you identify a new package you want to use,
# Just update the list below and re-run this command.
### Windows Command (don't include the #):
# py -m pip install --upgrade pip setuptools wheel
# py -m pip install --upgrade -r requirements.txt
### Mac/Linux Command (don't include the #):
# python3 -m pip install --upgrade pip setuptools wheel
# python3 -m pip install --upgrade -r requirements.txt
# When you identify a new package you want to use,
# Just update the list below and re-run the install command.
# ======================================================
# STEP D: VS CODE - Select Interpreter
# ======================================================
# VS Code needs our populated .venv to interpret our files correctly.
# To set the VS Code Interpreter:
# 1. Open the Command Palette: Press Ctrl+Shift+P (Windows/Linux) or Cmd+Shift+P (Mac).
# 2. Search for "Python: Select Interpreter":
# 3. Type Python: Select Interpreter in the Command Palette search bar and select it from the dropdown.
# 4. Choose an Interpreter - A list of available Python environments will appear.
# Look for the local .venv option.
# 5. Once selected, check the Python version displayed
# in the bottom-left corner of the VS Code window in the status bar.
# ======================================================
# COMMON STANDARD LIBRARY MODULES (NO INSTALL REQUIRED)
# ======================================================
# These modules are built into Python and do NOT need to be installed.
# They are available automatically when using Python.
# DO NOT UNCOMMENT THESE - THEY ARE ALREADY INCLUDED.
# json - For handling JSON data
# csv - For reading/writing CSV files
# datetime - For handling dates and times
# logging - For structured logging in Python
# os - For interacting with the operating system (e.g., file paths, environment variables)
# sys - For accessing system-specific parameters and functions
# pathlib - For working with filesystem paths
# sqlite3 - For working with SQLite databases (built into Python)
# time - For working with time-based functions
# random - For generating random numbers
# re - For regular expressions and pattern matching
# ======================================================
# ESSENTIAL EXTERNAL TOOLS - UNCOMMENT ONLY WHAT THE PROJECT NEEDS
# ======================================================
# Uncomment needed packages below and add more as required for the project.
# Up-to-date package management tools
pip
setuptools
wheel
# Easy logging for monitoring code execution
loguru
# Environment variables management
python-dotenv
# ======================================================
# TEXT-TO-SPEECH
# ======================================================
# Offline text-to-speech library for Python (1-15 MB)
# pyttsx3
# ======================================================
# JUPYTERLAB (OPTIONAL, NEEDED FOR NOTEBOOKS)
# ======================================================
# Next-generation web-based interactive development environment for Jupyter notebooks, code, and data.
# Offers a more feature-rich and versatile experience than the classic Jupyter Notebook. (60-70 MB)
jupyterlab
# Core IPython package that provides an enhanced interactive Python shell (10-15 MB).
ipython
# Core Jupyter functionality required for running notebooks in VS Code (50-60 MB).
jupyter
# Kernel interface for Jupyter notebooks (required for proper kernel registration)
ipykernel
# Interactive widgets (often used in notebooks).
ipywidgets
# ======================================================
# DATA STORAGE AND RETRIEVAL
# ======================================================
# SQLite database support (built into Python, no install needed)
# DuckDB: A lightweight, serverless database for Python (5-10 MB).
duckdb
# ORM for SQL databases (~10 MB) when sqlite3 is not enough
#sqlalchemy
# ======================================================
# DATA ANALYSIS
# ======================================================
# Numerical computations and arrays (20-30 MB)
#numpy
# Data manipulation and analysis (built on numpy, 10-20 MB)
pandas
# High-performance DataFrame library for large datasets (Rust-based, fast) (~5-10 MB)
#polars
# ======================================================
# VISUALIZATION
# ======================================================
# Core library for creating visualizations (~30 MB)
matplotlib
# Statistical data visualization library built on matplotlib (~2-5 MB)
seaborn
# Interactive plotting library, often used with Shiny apps (~20-25 MB)
#plotly
# ======================================================
# CONTINUOUS INTELLIGENCE AND INTERACTIVE ANALYTICS
# ======================================================
# Shiny framework for Python applications (~5-10 MB)
#shiny
# ======================================================
# KAFKA STREAMING MESSAGE BROKER INTEGRATION
# ======================================================
# Apache Kafka Python client (~1 MB)
#kafka-python-ng
# ======================================================
# STREAM PROCESSING - CAN BE VERY LARGE
# ======================================================
# Streaming data processing framework (~200-250 MB)
# Python compatibility varies by Spark version. Check the official documentation.
# Currently requires Python <3.12, e.g. 3.11
#pyspark<4.0.0
# Alternative lightweight parallel computing framework
#dask
# ======================================================
# MACHINE LEARNING (ML) and STATISTICAL MODELING
# ======================================================
# Statistical modeling and inference (~10-15 MB)
# statsmodels
# Core ML library with flexible APIs (~40-50 MB)
#scikit-learn
# ======================================================
# NATURAL LANGUAGE PROCESSING (NLP) - CAN BE VERY LARGE
# ======================================================
# NLP library for text processing (~10 MB core library; downloading corpora can add ~1 GB).
#nltk
# SpaCy: A powerful NLP library ~50 MB (core library; language models + ~300 MB or more per model).
# Currently requires Python <=3.12
#spacy<3.7
# ======================================================
# SENDING ALERTS via Simple Email or SMS Text using Gmail
# ======================================================
# These packages allow sending email or SMS alerts from Python.
# They require additional configuration before use.
# See setup instructions at:
# https://pypi.org/project/dc-mailer/ or https://github.com/denisecase/dc-mailer
# https://pypi.org/project/dc-texter/ or https://github.com/denisecase/dc-texter
# Uncomment the necessary package(s) to send email or SMS alerts.
# dc-mailer
# dc-texter