Debugging:
J(ϴ) should decrease after each iteration of running the training set.
i.e
by checking the J(ϴ) should decrease after each iteration to see if the
algorithm is correct.
Experience:
Rather rely on graph to see the convergence instead of counting on
automatic convergence test.
If with each iteration, the J(ϴ) is increasing, which possibly the learing rate
⍺ is too large, which over shoots.
Remember:
ϴJ = ϴi + ⍺*deferential(J(ϴ))
Choosing the learning rate ⍺:
If ⍺ is too small: slow convergence.
If ⍺ is too large: J(ϴ) may not decrease after each iteration, or might not
converge. Slow convergence also possible.
Plot the graph to debug.
Choose the ⍺ with 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, ...
increase the learning rate of 3 folds.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.