Machine Learning orientation
1
Introduction
I Machine learning: Shall we?
2
What is machine learning?
2.1
What is intelligence?
2.1.1
Definition of artificial intelligence sub domains
2.2
Is AI smarter than humans?
2.2.1
Thinking, fast and slow
(Kahneman
2011
)
2.3
Comparisons between AI and humans
2.3.1
Breast cancer detection
2.3.2
Working together: Lung cancer detection
2.3.3
ImageNet Large Scale Visual Recognition Challenge (ILSVRC)
2.3.4
AlphaGo Zero
2.4
ML models with bias
2.5
Attacks on ML models
2.5.1
Adding noise to image leads to misclassification
2.5.2
But what about attacks on human perception?
2.5.3
Model Hacking ADAS to Pave Safer Roads for Autonomous Vehicles
3
Outlook
3.1
Development of life
3.1.1
When will superhuman AI come, and will it be good?
3.1.2
AI aftermath scenario
3.2
Data religion: Dataism
4
Discussion points
4.1
Researcher stops his work due to ethical concerns
4.2
Career: Oxford seeks AI ethics professor
4.3
Shaping Europe’s digital future: Commission presents strategies for data and Artificial Intelligence
II Machine learning fundamentals
5
Machine learning fundamentals
6
ML project process
6.1
Identify if ML is suited to fulfill need
6.2
Gather data
6.2.1
What kind of data can be used?
6.2.2
How much data is necessary?
6.2.3
Which data is useful?
6.3
Exploratory and quantitative data analysis
6.3.1
Example for exploratory and quantitative data analysis
6.3.2
Visualizations for Categorical Data: Exploring the OkCupid Data
6.4
Feature engineering
6.4.1
Encoding Categorical Predictors
6.4.2
Engineering numeric features
6.4.3
Feature importance
6.5
Model fit
6.6
Model tuning
6.6.1
Metrics
7
Machine learning types
7.1
Supervised learning
7.1.1
Self supervised learning
7.2
Unsupervised learning
7.2.1
Discovering clusters
7.2.2
Discovering latent factors
7.3
Reinforcement learning
7.3.1
Elements of reinforcement learning
7.3.2
RL algorithms
7.3.3
Example self driving car MIT
8
Local examples
8.1
Master Autonomous Driving
8.2
University Suttgart: Indoor-Ortung mit Mobilfunk
8.3
Bionic Learning Network
8.4
AI translation
8.5
A Deep Learning Approach to Antibiotic Discovery
8.6
BBC: Artificial intelligence-created medicine to be used on humans for first time
8.7
Outbreak risk software
8.8
Neuronale Netze ermöglichen autonome Steuerung von Kathetern
8.9
Fundamental limits from chaos on instability time predictions in compact planetary systems
8.9.1
Comparison between SPOCK and previous models
8.10
Machine Learning Algorithms and Global Optimization Methods for SPICE Model Parameter Extraction
9
ML algorithms
9.1
Linear regression
9.1.1
Example for linear regression
9.2
Logistic regression
9.2.1
Python example logistic regression
9.3
Tree based methods
9.3.1
Splitting metrics
9.3.2
Ensembles
9.3.3
Random forest
9.3.4
Boosted trees
9.4
Support Vector Machine (SVM) TBD
9.4.1
Kernels
9.4.2
Python example for SVM
9.5
Neural networks
9.5.1
Convolutional Neural Network (CNN) TBD
9.5.2
RNN TBD
9.5.3
GANs
9.6
A Gentle Introduction to CycleGAN for Image Translation
9.6.1
Examples for GANs
9.7
Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
9.8
Transformers TBD
9.8.1
Example transformer for time series forecasting
10
Food for the algorithms: Data
10.1
NLP
10.2
Dataset search from Google
III Explainable ML
11
Explainable ML tbd
11.1
Method: Layer-Wise Relevance Propagation
11.2
Method: SpRay
11.3
Method: Lime tbd
11.4
alibi tbd
11.5
tf-explain tbd
11.6
keras-salient-object-visualization
11.6.1
Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
11.6.2
VisualBackProp: efficient visualization of CNNs
IV ML online resources
12
ML online courses
12.1
Coursera
12.2
Udemy
12.3
DataCamp
12.4
Udacity
12.4.1
Example for self-driving car course project
12.5
FastAI
12.6
Kaggle Courses
12.7
Full Stack Deep Learning
13
ML online resources
13.1
In-depth introduction to machine learning in 15 hours of expert videos
13.1.1
An Introduction to Statistical Learning
13.2
The learning machine
13.3
DeepAI: The front page of A.I.
13.4
TensorFlow tutorials
13.4.1
MIT 6.S191 Introduction to Deep Learning
13.5
Embedding Projector
13.6
Tensorboard playground
13.7
Empowering companies to jumpstart AI and generate real-world value
14
ML online books
14.1
Neural Networks and Deep Learning
14.2
Deep Learning
V Examples from Kaggle
15
Examples in Kaggle
16
Melbourne University AES/MathWorks/NIH Seizure Prediction
16.1
Winning solution (1st)
16.1.1
Alex / Gilberto models
16.1.2
Feng models
16.1.3
Andriy models
16.1.4
Code on GitHub
16.2
Solution(4th place)
16.2.1
Pre-processing
16.2.2
Features
16.2.3
Model
16.2.4
GitHub code
17
Bosch Production Line Performance
17.1
1st place solution
17.1.1
Data exploration
17.1.2
Hand crafted features
17.1.3
Hardware
17.2
3rd place solution TBD
17.3
8th place solution with GitHub
17.3.1
Overall architecture
17.3.2
Input data sets
17.3.3
Ensembling
17.3.4
Features
17.3.5
Validation method
17.3.6
Software
17.3.7
Code on GitHub
18
Corporación Favorita Grocery Sales Forecasting
18.1
1st place solution
18.2
4th-Place Solution Overview
18.3
5th Place Solution
19
Severstal: Steel Defect Detection
20
Lyft 3D Object Detection for Autonomous Vehicles
20.1
3rd place solution
21
APTOS 2019 Blindness Detection
21.1
1st place solution summary
22
Predicting Molecular Properties
22.1
#1 Solution - hybrid
22.1.1
Overall architecture
22.1.2
Ensembling
22.1.3
Hardware
22.1.4
Software
22.1.5
Code on GitHub
22.2
#2 solution 🤖 Quantum Uncertainty 🤖
22.2.1
Overall architecture
22.2.2
Input features and embeddings
22.2.3
Data augmentation
22.2.4
Ensembling
22.2.5
Hardware
22.2.6
Software
22.2.7
Code on GitHub
VI Real world example
23
Real world example
23.1
Subject of the project
Depending from where you were looking:
Looking from the perspective of machine learning expert
23.2
Project phases
The main project phases are:
After data gathering iteration is trump
23.2.1
Feature engineering
23.3
Algorithm selection
23.3.1
Logistic regression
23.3.2
Tree based
23.3.3
Support Vector Machine (SVM) TBD
23.4
Performance measurement
23.4.1
Sensitivity and specificity
23.4.2
Receiver operating characteristic (ROC)
23.5
Confusion matrix and receiver operating characterstic (ROC) for pulse
23.5.1
Receiver operating characterstic (ROC) and probability density plots
23.6
Create augmented labeled data
23.6.1
Features of time signals
23.7
Features generated
23.7.1
Analysis of generated features
23.7.2
Dynamic time warp (DTW) for signal
23.8
Algorithm
23.9
Confusion matrix results logistic regression for measured data
23.9.1
ROC results for measured data
23.10
Several algorithms results for SNR = 18dB
23.10.1
ROC results for SNR 18dB
23.11
Calculation of return of invest (ROI)
23.11.1
Calculation of ML project invest
23.11.2
Calculation of ML profit
23.11.3
Resulting ROI
23.12
Compare models for SNR = 18dB
23.13
Optimize ML hyper parameter
VII Cloud-based machine learning
24
Cloud-based machine learning
VIII Kaggle Survey
25
Kaggle survey introduction
25.1
Kaggle survey details
25.2
Purpose
25.3
Navigation and handling
26
Results
26.1
Survey participants education level
26.2
Who uses which algorithm
26.3
Machine learning experience and algorithms
26.4
Experience and new algorithms
26.5
Role of participants
26.6
Company size
26.7
Company incorporation of machine learning
26.8
Favourite media sources on data science topics
26.9
Favourite online course platform
26.10
Favourite data analyzing tool
26.11
Experience in data analysis coding
26.12
Favourite integrated development environments (IDE’s)
26.13
Favourite hosted notebook products
26.14
Favourite programming languages
26.15
Recommended entry programming language
26.16
Favourite data visualization libraries or tools
26.17
Favourite specialized hardware
26.18
Favourite machine learning frameworks
26.19
Favourite cloud computing platforms
26.20
Favourite big data / analytics products
26.21
Favourite automated machine learning tools (or partial AutoML tools)
References
Published with bookdown
Machine learning orientation
Chapter 8
Local examples