Probability for Machine Learning

Hi Folks !! In this post i will discuss about the tricks and tips that i use to solve questions based on probability and i will also discuss about where the concept of probability is used in Statistics and ML.

Note: [ This post is on probability and probability only and not on ML Algorithms that uses probability. But I assure you, after reading this post you will feel confident in reading and understanding all the Probability based ML -Algo’s like — {Naive Bayes | Bayesian Statistics etc .. } or in Stats like — {Binomial Probability Distribution etc ]. So without wasting time, lets get started .

Basics

Formula => P(Event) = Favourable Outcomes / Total Possible Outcomes . Lets look at some examples:

1. Problem: Throwing a Dice (1 time ) — Means [1,2,3,4,5,6] ie. total possible outcomes = 6. What is the probability of getting 5 on throwing a dice ? Ans : 1 / 6 .
2. Problem: Tossing a Coin (1 time) — Means [Heads , Tails ] ie. Total possible outcomes = 2. What is the probability of getting a Head or Tail on tossing a coin? Ans: p(H)= 1/2 , p(T) =1/2
3. Problem: Bucket with different colour balls and you are selecting a particular colour ball — Means total possible outcomes = number of balls in bucket . A bucket contains 10 green balls 7 blue balls and 3 white balls. You pick up a ball from basket (1 time) . What is the probability of getting a green ball ? Ans : 10 / (10 + 7 + 3) = 0.5

Now lets make this thing a little interesting !!!

What if i say, Throw 1 dice 10 times and find the probability of getting number ‘5’ exactly 7 times . Or Throw 5 dice together and find the probability of getting a sum 25 ?? [ Answer it in the comments section :) ] Tough huh !! Not really … To answer these questions you need to know few concepts . Lets look at them closely —

Types of Events

Mutually Exclusive events

1. Two events are mutually exclusive if they do not occur at same time .
2. Example : Head or Tail cannot occur at same time on tossing a coin.
3. Question: Two dice are rolled what is the probability that the sum of numbers rolled is either 5 or 9 ? Ans : P(AUB) = P(A) + P(B)

Independent Events

1. Two events are independent when the result of one event does not affect the other . They both are pure independent events .
2. Example: Tossing a coin 2 times one after the other are independent events .
3. Question: two dice are thrown one after the other . what is a probability of getting an even on both dice ? Ans : P(A)*P(B) = 3/6*3/6

Dependent Events

1. If the result of one event affects the result of other then both events are dependent.
2. Example: A basket has 10 balls (5 Blue and 5 White) and you select the first ball and keep it aside and then you select another ball from the basket .
3. Now, lets make a question out of point 2 . what is the P of getting blue both times ? Ans: P(A) * P(B,once A occurred ) = 5/10*4/9
4. P(AUB) = P(A)*P(B, once A has occurred)

Conditional Probability

1. This is based on the concept of dependent events , where probability that an event A takes place will depend on another event B.
2. Conditional Probability of an Event B given A [represented by=> P(B/A)]
3. Formula => P(B/A) = P(A and B intersection) / P(A)
4. Example: Suppose, I wrote 2 post about “probability” on Hackernoon and found that 25% of viewers read both of my post and 42% of viewers read only my 1st post .Now, What percent of those viewers who read my 1st post also read my 2nd post? ANS : .25/.42 = P(1st intersection 2nd) / P(1st) = 60% = P(2nd/1st) ( I hope you got this one ).

Bayes Theorem

1. It describes the probability of an event, based on prior knowledge of conditions that might be related to the event.
2. Formula : P(A/B) = P(B/A).P(A)/P(B)

Bayes theorem and Conditional Probability question style are very much inter-related. What i mean is that you can easily convert a Bayes theorem question to a one on Conditional probability and Vice versa by just changing 1 line of the entire question . Lets Look an Example :

Q. There are 2 basket with balls- BASKET-A and BASKET-B . Each basket contains 3 blue balls and 2 green balls . You select 1 ball from a Basket .What’s the probability of selecting a Blue ball ? (Conditional Probability Question)

Ans : Here, The Probability of picking Basket-A = 1/2 . Probability of picking Basket-B = 1/2 . Probability of selecting a Blue ball from Basket-A = 3/5 . Probability of selecting a Blue ball from Basket-B= 3/5 .

P( Selecting Blue Ball ) = P(blue Ball, Basket-A is selected first ) + P(blue Ball , Basket-B is selected first )

Final Ans = 3/5*1/2 + 3/5*1/2 (I hope you got this)

Converting the above conditional Probability Question into a Bayes !!

Q. There are 2 basket with balls- BASKET-A and BASKET-B . Each basket contains 3 blue balls and 2 green balls . You select 1 ball from a Basket and found that it’s a Blue Ball. What is the probability that its from Basket-A? (Bayes Question)

P(Basket-A | Blue Ball )= P(Blue Ball | Basket-A)* P(Basket-A)/P(Blue Ball)

P(Blue Ball ) = 3/5*1/2 + 3/5*1/2 (Calculated above in question)

P(Basket-A) = 1/2

P(Blue Ball | given Basket-A is selected ) = 1/2*3/5

Final Answer ( Simply put values in formula )= (3/10 * 1/2 )/(6/10)

You can also visualise the entire scenario by making a probability tree. (The image at the top) and then calculate evertyhing .

Visualisation Tree for Bayes Theorem

I hope you liked my post . Please give it a clap. It would encourage me to write more . If you are new to ml or Data Science, feel free to read my other post on machine learning by clicking on the link below .

Evaluating ML Models

Once Again, thank you for reading my post and don’t forget to Clap Share and Follow ..

Probability for Machine Learning was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.