Python for AI and Data Analysis

1 ... Introduction ... 13

1.1 ... Future Skill: Programming ... 13

1.2 ... Python ... 14

1.3 ... Artificial Intelligence ... 15

1.4 ... Machine Learning ... 16

1.5 ... Programming with the Help of AI ... 17

1.6 ... Prompt Engineering ... 19

2 ... Getting Started with Python ... 21

2.1 ... Installing Python on Windows ... 21

2.2 ... Installing Python on macOS and Linux ... 22

2.3 ... Using Python in Interactive Mode ... 22

2.4 ... Python Scripts ... 23

2.5 ... Visual Studio Code and Integrated Development Environments ... 23

2.6 ... Jupyter Notebooks ... 25

3 ... Basics of the Python Language ... 29

3.1 ... Variables and Data Types ... 30

3.2 ... Comments ... 31

3.3 ... Functions ... 31

3.4 ... Conditionals with "if," "elif," and "else" ... 32

3.5 ... Comparison Operators ... 33

3.6 ... Numbers ... 34

3.7 ... The "while" Loop ... 35

3.8 ... The "for" Loop ... 36

3.9 ... More on "print" ... 37

3.10 ... Lists, Sets, Tuples, and Dictionaries ... 38

3.11 ... "for" Loop for Lists and More ... 39

3.12 ... Sample Program: Counting Words ... 41

3.13 ... Writing Your Own Functions ... 44

3.14 ... Pythonic Code ... 47

3.15 ... Importing Modules and Installing Packages with "pip" ... 52

3.16 ... Virtual Environments ... 54

4 ... Working with Files ... 57

4.1 ... Reading and Writing Text Files ... 58

4.2 ... Comma-Separated Values Files ... 60

4.3 ... Managing Files ... 64

4.4 ... Example: Text Analysis ... 66

4.5 ... Excel Files ... 68

4.6 ... Image Files ... 70

4.7 ... JSON Files ... 73

4.8 ... XML Files ... 77

4.9 ... Configuration Files ... 78

5 ... Data Analysis ... 81

5.1 ... NumPy ... 81

5.2 ... Pandas ... 83

5.3 ... Loading Data from Files into Pandas DataFrames ... 87

5.4 ... Data Cleaning with Pandas ... 90

5.5 ... Calculations and Analyses with Pandas ... 99

5.6 ... Merging Data from Multiple Sources ... 104

6 ... Visualizations with Matplotlib ... 113

6.1 ... Creating Plots ... 113

6.2 ... Design Options ... 115

6.3 ... Subplots: Multiple Plots in a Single Figure ... 117

6.4 ... Line Charts, Bar Charts, and More ... 118

6.5 ... Creating Charts from DataFrames ... 121

6.6 ... Interactive Charts ... 124

6.7 ... Zooming and Scrolling ... 125

7 ... Machine Learning and Artificial Intelligence ... 131

7.1 ... Predicting Numbers Using Linear Regression ... 133

7.2 ... Linear Regression with Multiple Factors ... 135

7.3 ... Classification Using Logistic Regression ... 139

7.4 ... Decision Trees and Random Forests ... 141

7.5 ... K-Nearest Neighbors ... 144

7.6 ... Support Vector Machines ... 148

7.7 ... Training Data, Test Data, and Model Evaluation ... 151

7.8 ... Clustering (Unsupervised Learning) ... 158

8 ... AI in Action: Text and Image Analysis ... 163

8.1 ... AI for Text and Language ... 163

8.2 ... Text Analysis and Word Clouds ... 164

8.3 ... Text Preprocessing ... 166

8.4 ... Sentiment Analysis ... 170

8.5 ... Recognizing Entities in Text: Named Entity Recognition ... 175

8.6 ... Transfer Learning ... 178

8.7 ... AI for Images ... 181

8.8 ... Image Preprocessing: Grayscale Conversion and More ... 182

8.9 ... Detecting Edges and Contours in Images ... 185

8.10 ... Classic Machine Learning Methods for Images ... 189

8.11 ... Image Classification with Deep Learning ... 195

9 ... Using APIs ... 199

9.1 ... API Requests with "requests" ... 200

9.2 ... API Access with Special SDKs ... 205

9.3 ... Visualizing and Analyzing API Data ... 206

9.4 ... ChatGPT API ... 212

10 ... Using Python on the Web ... 217

10.1 ... HTML ... 218

10.2 ... Flask: A Python Web Server ... 220

10.3 ... Interactive Web Tools with Streamlit ... 222

10.4 ... Reading Web Content with Beautiful Soup ... 225

10.5 ... Remotely Controlling the Browser with Selenium ... 228

10.6 ... Sending Emails and Messenger Messages ... 231

11 ... Databases ... 235

11.1 ... The SQL Language and the SQLite Console ... 236

11.2 ... Creating Tables with CREATE TABLE ... 238

11.3 ... Querying, Inserting, Updating, and Deleting with SELECT, INSERT, UPDATE, and DELETE ... 240

11.4 ... Accessing an SQLite Database with Python ... 242

11.5 ... Pandas DataFrame: Reading and Writing Data from Databases ... 244

12 ... Automating Routine Tasks ... 247

12.1 ... Retrieving Data via the API and Storing It in a Database ... 247

12.2 ... Creating Charts from the Database and Save as Image Files ... 250

12.3 ... Sending an Email When There's Something New ... 252

12.4 ... Sending Screenshots of Web Pages via Messenger ... 255

12.5 ... Images: Reducing File Sizes, Finding Locations, and Cleaning Up ... 258

12.6 ... PDF: Splitting, Merging, and Consolidating Files ... 261

12.7 ... Recognizing Text on Business Card Images Using OCR and Saving to Your Phone's Address Book ... 263

12.8 ... Creating Invoices, Mail Merges, and Other Documents ... 265

12.9 ... Running Python Scripts on a Schedule ... 268

12.10 ... Logging in Automations ... 271

... The Author ... 275

... Index ... 27