• Machine Learning orientation
  • 1 Introduction
  • I Machine learning: Shall we?
  • 2 What is machine learning?
    • 2.1 What is intelligence?
      • 2.1.1 Definition of artificial intelligence sub domains
    • 2.2 Is AI smarter than humans?
      • 2.2.1 Thinking, fast and slow
      • 2.2.2 Predictions
      • 2.2.3 Decisions
    • 2.3 Comparisons between AI and humans
      • 2.3.1 Breast cancer detection
      • 2.3.2 Working together: Lung cancer detection
      • 2.3.3 ImageNet Large Scale Visual Recognition Challenge (ILSVRC)
      • 2.3.4 AlphaGo Zero
    • 2.4 ML models with bias
      • 2.4.1 What is the relation between bias in machine learning and priors of Bayes theorem?
      • 2.4.2 How to avoid bias
      • 2.4.3 Human bias
    • 2.5 Attacks on ML models
      • 2.5.1 Adding noise to image leads to misclassification
      • 2.5.2 But what about attacks on human perception?
      • 2.5.3 Model Hacking ADAS to Pave Safer Roads for Autonomous Vehicles
    • 2.6 Measuring the Algorithmic Efficiency of Neural Networks TBD
  • 3 ML application examples
    • 3.1 AlphaFold: a solution to a 50-year-old grand challenge in biology
      • 3.1.1 Results of 2020 CASP
      • 3.1.2 Why does it matter?
      • 3.1.3 CASP details
    • 3.2 AI translation by DeepL
    • 3.3 Artificial intelligence detects myocardial infarctions in the ECG more reliably than cardiologistsn
      • 3.3.1 First in Germany: Artificial intelligence recognizes COVID-19 in clinical routine
    • 3.4 A Deep Learning Approach to Antibiotic Discovery
    • 3.5 Fundamental limits from chaos on instability time predictions in compact planetary systems
      • 3.5.1 Comparison between SPOCK and previous models
    • 3.6 Solution of the Schrödinger equation using deep neural networks
    • 3.7 Global Optimization Methods for SPICE Model Parameter Extraction
    • 3.8 Fraud detection based on mouse movement images
    • 3.9 Latent diffusion models (LDM)
      • 3.9.1 Image generation from text with LDM (text2img)
      • 3.9.2 Super resolution
      • 3.9.3 Inpainting
      • 3.9.4 Imagic: Text-Based Real Image Editing with Diffusion Models
      • 3.9.5 Stable diffusion under the hood
    • 3.10 How many yards will an NFL player gain after receiving a handoff?
    • 3.11 Predictive Maintenance for the elevator and escalator industry TBD
    • 3.12 BBC: Artificial intelligence-created medicine to be used on humans for first time
    • 3.13 Disease outbreak risk software
    • 3.14 Neural networks enable autonomous navigation of catheters
    • 3.15 Bosch FLEXIDOME IP starlight 8000i
    • 3.16 Demonstration of computer vision system “thumbs up”
      • 3.16.1 Run demonstration on Jetson Nano
    • 3.17 Master Autonomous Driving
    • 3.18 University Suttgart: Indoor-Ortung mit Mobilfunk
  • 4 Data Ethics
    • 4.1 Topics in data ethics
      • 4.1.1 Recourse and accountabilty
      • 4.1.2 Feedback loops
      • 4.1.3 Bias
    • 4.2 Identify and adressing ethical issues
      • 4.2.1 Analyze a planned project
      • 4.2.2 Process to implement
      • 4.2.3 Diversity
    • 4.3 Data Ethics consequences
      • 4.3.1 Researcher stops his work due to ethical concerns
      • 4.3.2 Career: Oxford seeks AI ethics professor
      • 4.3.3 Shaping Europe’s digital future: Commission presents strategies for data and Artificial Intelligence
  • 5 Strategies for machine learning
    • 5.1 The HOOD-ST2C ML strategy
    • 5.2 Management ML strategy: 7 steps for a successful ML project
    • 5.3 Management ML strategy: Data Project Checklist
    • 5.4 Project management ML strategy: The Drivetrain Approach
      • 5.4.1 Recommendation system
      • 5.4.2 Exercise: Optimizing lifetime customer value
    • 5.5 Developer ML strategy TBD
  • 6 Outlook of ML future
    • 6.1 Development of life
      • 6.1.1 When will superhuman AI come, and will it be good?
      • 6.1.2 AI aftermath scenario
    • 6.2 Data religion: Dataism
  • II Machine learning fundamentals
  • 7 ML project process
    • 7.1 Identify if ML is suited to fulfill need
    • 7.2 Gather data
      • 7.2.1 What kind of data can be used?
      • 7.2.2 How much data is necessary?
      • 7.2.3 Which data is useful?
    • 7.3 Data analysis
      • 7.3.1 Example for exploratory and quantitative data analysis
      • 7.3.2 Visualizations for Categorical Data: Exploring the OkCupid Data
    • 7.4 Feature engineering
      • 7.4.1 Encoding Categorical Predictors
      • 7.4.2 Engineering numeric features
      • 7.4.3 Feature importance
    • 7.5 Model fit
    • 7.6 Model tuning
      • 7.6.1 Metrics
    • 7.7 Model deployment
  • 8 Machine learning types
    • 8.0.1 Machine learning types by data type
    • 8.0.2 Machine learning types by data content
    • 8.1 Supervised learning
      • 8.1.1 Self supervised learning
    • 8.2 Unsupervised learning
      • 8.2.1 Discovering clusters
      • 8.2.2 Discovering latent factors
    • 8.3 Reinforcement learning
      • 8.3.1 Elements of reinforcement learning
      • 8.3.2 RL algorithms
      • 8.3.3 Example self driving car MIT
  • 9 ML algorithms
    • 9.1 Linear regression
      • 9.1.1 Example for linear regression
    • 9.2 Logistic regression
      • 9.2.1 Python example logistic regression
    • 9.3 Tree based methods
      • 9.3.1 Splitting metrics
      • 9.3.2 Ensembles
      • 9.3.3 Random forest
      • 9.3.4 Boosted trees
    • 9.4 Support Vector Machine (SVM) TBD
      • 9.4.1 Kernels
      • 9.4.2 Python example for SVM
    • 9.5 Neural networks
      • 9.5.1 Geometric Intuition for Training Neural Networks
      • 9.5.2 Convolutional Neural Network (CNN) TBD
      • 9.5.3 RNN TBD
      • 9.5.4 GANs
    • 9.6 A Gentle Introduction to CycleGAN for Image Translation
      • 9.6.1 Examples for GANs
    • 9.7 Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
      • 9.7.1 Pix2pix framework
    • 9.8 Transformers TBD
      • 9.8.1 Example transformer for time series forecasting
    • 9.9 Naive Bayes Classifier
      • 9.9.1 Gaussian Naive Bayes¶
  • 10 Food for the algorithms: Data
    • 10.1 NLP
    • 10.2 Dataset search from Google
  • III Explainable machine learning
  • 11 Explainable machine learning
    • 11.1 Method: Layer-Wise Relevance Propagation
    • 11.2 Method: SpRay
    • 11.3 Method: Salient-object-visualization
      • 11.3.1 Tool: keras-salient-object-visualization
      • 11.3.2 Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
    • 11.4 Tool: Lime
    • 11.5 Tool: tf-explain
    • 11.6 Tool: alibi TBD
  • IV ML online resources
  • 12 ML online courses
    • 12.1 Coursera
    • 12.2 Udemy
    • 12.3 DataCamp
    • 12.4 Udacity
      • 12.4.1 Example for self-driving car course project
    • 12.5 fast.ai
      • 12.5.1 fast.ai Book
    • 12.6 Kaggle Courses
    • 12.7 Full Stack Deep Learning
  • 13 ML online resources
    • 13.1 In-depth introduction to machine learning in 15 hours of expert videos
      • 13.1.1 An Introduction to Statistical Learning
    • 13.2 The learning machine
    • 13.3 DeepAI: The front page of A.I.
    • 13.4 TensorFlow tutorials
      • 13.4.1 MIT 6.S191 Introduction to Deep Learning
    • 13.5 Embedding Projector
    • 13.6 Tensorboard playground
    • 13.7 Empowering companies to jumpstart AI and generate real-world value
    • 13.8 TensorFlow, Keras and deep learning, without a PhD
    • 13.9 Neural Networks and Deep Learning
    • 13.10 Platform.ai: produce high-quality labels
  • 14 ML online books
    • 14.1 Neural Networks and Deep Learning
    • 14.2 Deep Learning
  • V Examples from Kaggle
  • 15 Examples in Kaggle
  • 16 PetFinder.my - Pawpularity Contest
    • 16.1 The winning solution
  • 17 Melbourne University AES/MathWorks/NIH Seizure Prediction
    • 17.1 Winning solution (1st)
      • 17.1.1 Alex / Gilberto models
      • 17.1.2 Feng models
      • 17.1.3 Andriy models
      • 17.1.4 Code on GitHub
    • 17.2 Solution(4th place)
      • 17.2.1 Pre-processing
      • 17.2.2 Features
      • 17.2.3 Model
      • 17.2.4 GitHub code
  • 18 Bosch Production Line Performance
    • 18.1 1st place solution
      • 18.1.1 Data exploration
      • 18.1.2 Hand crafted features
      • 18.1.3 Hardware
    • 18.2 3rd place solution TBD
    • 18.3 8th place solution with GitHub
      • 18.3.1 Overall architecture
      • 18.3.2 Input data sets
      • 18.3.3 Ensembling
      • 18.3.4 Features
      • 18.3.5 Validation method
      • 18.3.6 Software
      • 18.3.7 Code on GitHub
  • 19 Corporación Favorita Grocery Sales Forecasting
    • 19.1 1st place solution
    • 19.2 4th-Place Solution Overview
    • 19.3 5th Place Solution
  • 20 Rossmann Store Sales
    • 20.1 1st place solution
    • 20.2 3rd place solution
  • 21 Severstal: Steel Defect Detection
  • 22 Lyft 3D Object Detection for Autonomous Vehicles
    • 22.1 3rd place solution
  • 23 APTOS 2019 Blindness Detection
    • 23.1 1st place solution summary
  • 24 Predicting Molecular Properties
    • 24.1 #1 Solution - hybrid
      • 24.1.1 Overall architecture
      • 24.1.3 Ensembling
      • 24.1.4 Hardware
      • 24.1.5 Software
      • 24.1.6 Code on GitHub
    • 24.2 #2 solution 🤖 Quantum Uncertainty 🤖
      • 24.2.1 Overall architecture
      • 24.2.2 Input features and embeddings
      • 24.2.3 Data augmentation
      • 24.2.4 Ensembling
      • 24.2.5 Hardware
      • 24.2.6 Software
      • 24.2.7 Code on GitHub
  • VI Real world example
  • 25 Real world example
    • 25.1 Subject of the project
      • Depending from where you were looking:
      • Looking from the perspective of machine learning expert
    • 25.2 Project phases
      • The main project phases are:
      • After data gathering iteration is trump
      • 25.2.1 Feature engineering
    • 25.3 Algorithm selection
      • 25.3.1 Logistic regression
      • 25.3.2 Tree based
      • 25.3.3 Support Vector Machine (SVM) TBD
    • 25.4 Performance measurement
      • 25.4.1 Sensitivity and specificity
      • 25.4.2 Receiver operating characteristic (ROC)
    • 25.5 Confusion matrix and receiver operating characterstic (ROC) for pulse
      • 25.5.1 Receiver operating characterstic (ROC) and probability density plots
    • 25.6 Create augmented labeled data
      • 25.6.1 Features of time signals
    • 25.7 Features generated
      • 25.7.1 Analysis of generated features
      • 25.7.2 Dynamic time warp (DTW) for signal
    • 25.8 Confusion matrix results logistic regression for measured data
      • 25.8.1 ROC results for measured data
    • 25.9 Several algorithms results for SNR = 18dB
      • 25.9.1 ROC results for SNR 18dB
    • 25.10 Calculation of return of invest (ROI)
      • 25.10.1 Calculation of ML project invest
      • 25.10.2 Calculation of ML profit
      • 25.10.3 Resulting ROI
  • VII Cloud-based machine learning
  • 26 Cloud-based machine learning
  • VIII Kaggle Survey
  • 27 Kaggle survey introduction
    • 27.1 Kaggle survey details
    • 27.2 Purpose
    • 27.3 Navigation and handling
  • 28 Results
    • 28.1 Survey participants education level
    • 28.2 Who uses which algorithm
    • 28.3 Machine learning experience and algorithms
    • 28.4 Experience and new algorithms
    • 28.5 Role of participants
    • 28.6 Company size
    • 28.7 Company incorporation of machine learning
    • 28.8 Favourite media sources on data science topics
    • 28.9 Favourite online course platform
    • 28.10 Favourite data analyzing tool
    • 28.11 Experience in data analysis coding
    • 28.12 Favourite integrated development environments (IDE’s)
    • 28.13 Favourite hosted notebook products
    • 28.14 Favourite programming languages
    • 28.15 Recommended entry programming language
    • 28.16 Favourite data visualization libraries or tools
    • 28.17 Favourite specialized hardware
    • 28.18 Favourite machine learning frameworks
    • 28.19 Favourite cloud computing platforms
    • 28.20 Favourite big data / analytics products
    • 28.21 Favourite automated machine learning tools (or partial AutoML tools)
  • References
  • Published with bookdown

Machine learning orientation

23.1 1st place solution summary

https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/108065#latest-673088