For answer to this issue I have checked certain posts on the forum of Andrew’s course on Cousera, getting instructions as followed:

The point about the impact of regularization going down with increase in the number of samples also makes a lot of sense. So the key points:

  • regularization penalizes the higher order terms more as the θi for higher order terms need to be larger to have the same impact because each of the features are guaranteed to be less than ‘1’ because of normalization.
  • The impact of regularization decreases with the increase is sample size because the 1/m term in the regularization cost.

See details at https://class.coursera.org/ml-003/forum/thread?thread_id=3807#comment-12349

Advertisements