Below is a pdf with some exercises for that i use in my course on inverse problems.
Problem 0: Goal understand why it is attractive to use loglikelihoods over likelihoods.
Problem 1: An exact model with a single model parameter. This makes easy to plot.
Problem 2: A variant of problem 1 with an inexact model.
Problem 3: Goal: Show that it is easier to calculate derived quantities such as "expected value" of the posterior when you use mcmc.
Problem 4: A problem where it is easy to understand why is the likelihood called a likelihood.
Problem 5: A problem where you have to grasp many concepts.
Exercises 4 & 5 are probably the most fun.