package owl-opt

Owl Opt Library

The entry point of this library is Owl_opt.

Workflow

1. define a record type 'a t in some module Prms
2. apply ppx deriver @@deriving prms to type 'a t so that Prms has type Owl_opt.Prms.PT
3. pass Prms through your favourite algorithm functor to create an optimisation module O
4. define an objective function f that takes as input your record type 'a t
5. define initial parameters prms0
6. define learning rates lr with Owl_opt.Lr
7. initialise optimisation state s0 with O.init ~prms0 ~f
8. define stoping criterion function stop: state -> bool
9. minimise the optimisation session with O.min ~stop s0

Example

 module Prms = struct
    type 'a t = {a: 'a; b: 'a} [@@deriving prms]
 end
 (* make an Adam optimisation module for the parameter definition Prms *)
 module O = Owl_opt.D.Adam.Make (Prms)
 (* define the objective function *)
 let f prms = Owl.Algodiff.D.Maths.(l2norm' (y - ((prms.a *@ x) + prms.b))) 
 (* define initial parameters *)
 let prms0 = {a = Owl.Algodiff.D.Mat.gaussian 5 5; b = Owl.Algodiff.D.gaussian 5 1} 
 (* define fixed learning rate *)
 let lr = Owl_opt.Lr.(Fix 1E-4) 
 (* initialise an optimisation session *)
 let s0 = O.init ~prms0 ~f () 
 (* define stopping criteria: stop when function value is smaller than 1E-4 *)
 let stop s = O.(fv s) < 1E-4
 (* minimise objective function f *)
 let s = O.min ~stop ~beta1:0.99 ~beta2:0.999 ~lr f
 (* final objective function value *)
 let c = O.fv s
 (* final prms *)
 let prms = O.prms s 

Important modules

1. Parameters

• module Owl_opt.Prms and module type definition Owl_opt.Prms.PT

2. Learning rate

• module Owl_opt.Lr

3. Double-precision

• Vanilla gradient descent Owl_opt.D.Gd.Make

• Adam Owl_opt.D.Adam.Make

• Rmsprop Owl_opt.D.Rmsprop.Make

• Lbfgs (see Owl Opt Lbfgs)

4. Single-precision

• Vanilla gradient descent Owl_opt.S.Gd.Make

• Adam Owl_opt.S.Adam.Make

• Rmsprop Owl_opt.S.Rmsprop.Make