Machine Learning orientation
1
Introduction
I Machine learning: Shall we?
2
What is machine learning?
2.1
What is intelligence?
2.1.1
Definition of artificial intelligence sub domains
2.2
Is AI smarter than humans?
2.2.1
Thinking, fast and slow
2.2.2
Predictions
2.2.3
Decisions
2.3
Comparisons between AI and humans
2.3.1
Breast cancer detection
2.3.2
Working together: Lung cancer detection
2.3.3
ImageNet Large Scale Visual Recognition Challenge (ILSVRC)
2.3.4
AlphaGo Zero
2.4
ML models with bias
2.4.1
What is the relation between bias in machine learning and priors of Bayes theorem?
2.4.2
How to avoid bias
2.4.3
Human bias
2.5
Attacks on ML models
2.5.1
Adding noise to image leads to misclassification
2.5.2
But what about attacks on human perception?
2.5.3
Model Hacking ADAS to Pave Safer Roads for Autonomous Vehicles
2.6
Measuring the Algorithmic Efficiency of Neural Networks TBD
3
ML application examples
3.1
AlphaFold: a solution to a 50-year-old grand challenge in biology
3.1.1
Results of 2020 CASP
3.1.2
Why does it matter?
3.1.3
CASP details
3.2
AI translation by DeepL
3.3
Artificial intelligence detects myocardial infarctions in the ECG more reliably than cardiologistsn
3.3.1
First in Germany: Artificial intelligence recognizes COVID-19 in clinical routine
3.4
A Deep Learning Approach to Antibiotic Discovery
3.5
Fundamental limits from chaos on instability time predictions in compact planetary systems
3.5.1
Comparison between SPOCK and previous models
3.6
Solution of the Schrödinger equation using deep neural networks
3.7
Global Optimization Methods for SPICE Model Parameter Extraction
3.8
Fraud detection based on mouse movement images
3.9
Latent diffusion models (LDM)
3.9.1
Image generation from text with LDM (text2img)
3.9.2
Super resolution
3.9.3
Inpainting
3.9.4
Imagic: Text-Based Real Image Editing with Diffusion Models
3.9.5
Stable diffusion under the hood
3.10
How many yards will an NFL player gain after receiving a handoff?
3.11
Predictive Maintenance for the elevator and escalator industry TBD
3.12
BBC: Artificial intelligence-created medicine to be used on humans for first time
3.13
Disease outbreak risk software
3.14
Neural networks enable autonomous navigation of catheters
3.15
Bosch FLEXIDOME IP starlight 8000i
3.16
Demonstration of computer vision system “thumbs up”
3.16.1
Run demonstration on Jetson Nano
3.17
Master Autonomous Driving
3.18
University Suttgart: Indoor-Ortung mit Mobilfunk
4
Data Ethics
4.1
Topics in data ethics
4.1.1
Recourse and accountabilty
4.1.2
Feedback loops
4.1.3
Bias
4.2
Identify and adressing ethical issues
4.2.1
Analyze a planned project
4.2.2
Process to implement
4.2.3
Diversity
4.3
Data Ethics consequences
4.3.1
Researcher stops his work due to ethical concerns
4.3.2
Career: Oxford seeks AI ethics professor
4.3.3
Shaping Europe’s digital future: Commission presents strategies for data and Artificial Intelligence
5
Strategies for machine learning
5.1
The HOOD-ST2C ML strategy
5.2
Management ML strategy: 7 steps for a successful ML project
5.3
Management ML strategy: Data Project Checklist
5.4
Project management ML strategy: The Drivetrain Approach
5.4.1
Recommendation system
5.4.2
Exercise: Optimizing lifetime customer value
5.5
Developer ML strategy TBD
6
Outlook of ML future
6.1
Development of life
6.1.1
When will superhuman AI come, and will it be good?
6.1.2
AI aftermath scenario
6.2
Data religion: Dataism
II Machine learning fundamentals
7
ML project process
7.1
Identify if ML is suited to fulfill need
7.2
Gather data
7.2.1
What kind of data can be used?
7.2.2
How much data is necessary?
7.2.3
Which data is useful?
7.3
Data analysis
7.3.1
Example for exploratory and quantitative data analysis
7.3.2
Visualizations for Categorical Data: Exploring the OkCupid Data
7.4
Feature engineering
7.4.1
Encoding Categorical Predictors
7.4.2
Engineering numeric features
7.4.3
Feature importance
7.5
Model fit
7.6
Model tuning
7.6.1
Metrics
7.7
Model deployment
8
Machine learning types
8.0.1
Machine learning types by data type
8.0.2
Machine learning types by data content
8.1
Supervised learning
8.1.1
Self supervised learning
8.2
Unsupervised learning
8.2.1
Discovering clusters
8.2.2
Discovering latent factors
8.3
Reinforcement learning
8.3.1
Elements of reinforcement learning
8.3.2
RL algorithms
8.3.3
Example self driving car MIT
9
ML algorithms
9.1
Linear regression
9.1.1
Example for linear regression
9.2
Logistic regression
9.2.1
Python example logistic regression
9.3
Tree based methods
9.3.1
Splitting metrics
9.3.2
Ensembles
9.3.3
Random forest
9.3.4
Boosted trees
9.4
Support Vector Machine (SVM) TBD
9.4.1
Kernels
9.4.2
Python example for SVM
9.5
Neural networks
9.5.1
Geometric Intuition for Training Neural Networks
9.5.2
Convolutional Neural Network (CNN) TBD
9.5.3
RNN TBD
9.5.4
GANs
9.6
A Gentle Introduction to CycleGAN for Image Translation
9.6.1
Examples for GANs
9.7
Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
9.7.1
Pix2pix framework
9.8
Transformers TBD
9.8.1
Example transformer for time series forecasting
9.9
Naive Bayes Classifier
9.9.1
Gaussian Naive Bayes¶
10
Food for the algorithms: Data
10.1
NLP
10.2
Dataset search from Google
III Explainable machine learning
11
Explainable machine learning
11.1
Method: Layer-Wise Relevance Propagation
11.2
Method: SpRay
11.3
Method: Salient-object-visualization
11.3.1
Tool: keras-salient-object-visualization
11.3.2
Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
11.4
Tool: Lime
11.5
Tool: tf-explain
11.6
Tool: alibi TBD
IV ML online resources
12
ML online courses
12.1
Coursera
12.2
Udemy
12.3
DataCamp
12.4
Udacity
12.4.1
Example for self-driving car course project
12.5
fast.ai
12.5.1
fast.ai Book
12.6
Kaggle Courses
12.7
Full Stack Deep Learning
13
ML online resources
13.1
In-depth introduction to machine learning in 15 hours of expert videos
13.1.1
An Introduction to Statistical Learning
13.2
The learning machine
13.3
DeepAI: The front page of A.I.
13.4
TensorFlow tutorials
13.4.1
MIT 6.S191 Introduction to Deep Learning
13.5
Embedding Projector
13.6
Tensorboard playground
13.7
Empowering companies to jumpstart AI and generate real-world value
13.8
TensorFlow, Keras and deep learning, without a PhD
13.9
Neural Networks and Deep Learning
13.10
Platform.ai: produce high-quality labels
14
ML online books
14.1
Neural Networks and Deep Learning
14.2
Deep Learning
V Examples from Kaggle
15
Examples in Kaggle
16
PetFinder.my - Pawpularity Contest
16.1
The winning solution
17
Melbourne University AES/MathWorks/NIH Seizure Prediction
17.1
Winning solution (1st)
17.1.1
Alex / Gilberto models
17.1.2
Feng models
17.1.3
Andriy models
17.1.4
Code on GitHub
17.2
Solution(4th place)
17.2.1
Pre-processing
17.2.2
Features
17.2.3
Model
17.2.4
GitHub code
18
Bosch Production Line Performance
18.1
1st place solution
18.1.1
Data exploration
18.1.2
Hand crafted features
18.1.3
Hardware
18.2
3rd place solution TBD
18.3
8th place solution with GitHub
18.3.1
Overall architecture
18.3.2
Input data sets
18.3.3
Ensembling
18.3.4
Features
18.3.5
Validation method
18.3.6
Software
18.3.7
Code on GitHub
19
Corporación Favorita Grocery Sales Forecasting
19.1
1st place solution
19.2
4th-Place Solution Overview
19.3
5th Place Solution
20
Rossmann Store Sales
20.1
1st place solution
20.2
3rd place solution
21
Severstal: Steel Defect Detection
22
Lyft 3D Object Detection for Autonomous Vehicles
22.1
3rd place solution
23
APTOS 2019 Blindness Detection
23.1
1st place solution summary
24
Predicting Molecular Properties
24.1
#1 Solution - hybrid
24.1.1
Overall architecture
24.1.3
Ensembling
24.1.4
Hardware
24.1.5
Software
24.1.6
Code on GitHub
24.2
#2 solution 🤖 Quantum Uncertainty 🤖
24.2.1
Overall architecture
24.2.2
Input features and embeddings
24.2.3
Data augmentation
24.2.4
Ensembling
24.2.5
Hardware
24.2.6
Software
24.2.7
Code on GitHub
VI Real world example
25
Real world example
25.1
Subject of the project
Depending from where you were looking:
Looking from the perspective of machine learning expert
25.2
Project phases
The main project phases are:
After data gathering iteration is trump
25.2.1
Feature engineering
25.3
Algorithm selection
25.3.1
Logistic regression
25.3.2
Tree based
25.3.3
Support Vector Machine (SVM) TBD
25.4
Performance measurement
25.4.1
Sensitivity and specificity
25.4.2
Receiver operating characteristic (ROC)
25.5
Confusion matrix and receiver operating characterstic (ROC) for pulse
25.5.1
Receiver operating characterstic (ROC) and probability density plots
25.6
Create augmented labeled data
25.6.1
Features of time signals
25.7
Features generated
25.7.1
Analysis of generated features
25.7.2
Dynamic time warp (DTW) for signal
25.8
Confusion matrix results logistic regression for measured data
25.8.1
ROC results for measured data
25.9
Several algorithms results for SNR = 18dB
25.9.1
ROC results for SNR 18dB
25.10
Calculation of return of invest (ROI)
25.10.1
Calculation of ML project invest
25.10.2
Calculation of ML profit
25.10.3
Resulting ROI
VII Cloud-based machine learning
26
Cloud-based machine learning
VIII Kaggle Survey
27
Kaggle survey introduction
27.1
Kaggle survey details
27.2
Purpose
27.3
Navigation and handling
28
Results
28.1
Survey participants education level
28.2
Who uses which algorithm
28.3
Machine learning experience and algorithms
28.4
Experience and new algorithms
28.5
Role of participants
28.6
Company size
28.7
Company incorporation of machine learning
28.8
Favourite media sources on data science topics
28.9
Favourite online course platform
28.10
Favourite data analyzing tool
28.11
Experience in data analysis coding
28.12
Favourite integrated development environments (IDE’s)
28.13
Favourite hosted notebook products
28.14
Favourite programming languages
28.15
Recommended entry programming language
28.16
Favourite data visualization libraries or tools
28.17
Favourite specialized hardware
28.18
Favourite machine learning frameworks
28.19
Favourite cloud computing platforms
28.20
Favourite big data / analytics products
28.21
Favourite automated machine learning tools (or partial AutoML tools)
References
Published with bookdown
Machine learning orientation
19.1
1st place solution
The first place solution is described at
Kaggle discussion