## Markov Decision Processes

First let’s have a framework for what we would call randomness throughout time. Stochastic Procesess A Stochastic Process is defined as a collection of random variables $\{\ X(t)\}_{t \in T}\ = \{\ x_1 , x_2 , \dots , x_T \}\$ indexed by time (the subscript). When for all elements of this collection assume a value, we call each value in thi...

## Matrix Multiplication

Remember the good ol days when 6 x 5 easily made sense as adding 6 together with itself 5 times and whala you ended up with 30. Now you’re in college and things are hard :( Hopefully running through an example can give you a bit of a glimpse as to how and why we do matrix multiplication. How to Compute Matrix Multiplication Consider two Matric...

## Bayes Theorem and ROC

Conditional Probability Is the idea of understanding the probability an event A occurs given an event B. We formally say the Conditional Probability of A given B is defined as: $P(A | B ) = \frac{P(A \cap B)}{P(B)}$ Photo Courtesy of [“Conditional Probability and Independent Events”] Pictorially we consider the 3 ...

## Combinations

Since the concept of “n choose k” seems to appear a lot in my life I decided I would make a quick post explaining the intuition behind it. Let’s start with a simple example. Say we had a set of three greek characters representing the names of three friends, $F = { \alpha, \beta, \gamma }$ and we are interested in knowing how many uniquely pair...

## The Convolution Integral

Since signals are sets of data or information and systems process said data, we are interested in the analysis of systems. When we deal with a special type of system that contains the properties of linearity and time-invariance, we are able to construct methods of analysis that are extremely useful for Linear Time-invariant (LTI) systems. Fourie...

## Random Variables and Distributions

I hope this article serves as a basic introduction to the terminology of probability theory! Random Variables Considering that an experiment is a procedure that produces well defined outcomes, like taking a course and finishing with a certain grade letter, we see that a random variable is a function which maps random outcomes from experiments ...