.. _fit_methods:

FIT_METHODS
##############

A required section that defines how |popy| estimates the |fx| and |rx| model parameters given a |data_file|. 

The |fit_methods| section is required in the following scripts:-

* :ref:`tut_script`
* :ref:`fit_script`
* :ref:`mtut_script`
* :ref:`mfit_script`

|ie| Any script that estimates |pkpd| model parameters is required to have a |fit_methods| section.

.. _fit_methods_structure:

FIT_METHODS Structure
=======================

The |fit_methods| section is a list of fitting methods to be applied in order to estimate the |fx| and |rx| variables of a |pkpd| model, for example:-

.. code-block:: pyml

    FIT_METHODS:
        - <fitter1>: {}
        - <fitter2>: {}
        - <fitter3>: {}

This allows in this case <fitter1> to initialise the |fx| for <fitter2> and for <fitter2> to then initialise the |fx| for  <fitter3> |etc|.

In the long term it will be possible to apply multiple fitting methods, for example methods that globally initialise the |fx| followed by local fitting methods such as |joe| and maybe later more time consuming fitting approaches such as |imp|.

With the first release of |popy| the only available fitting method is |joe|.

.. _fit_methods_joe:

JOE Fitting Method
==========================

|joe| stands for |joe_long| and is |popy|'s approximate equivalent of |nonmem|'s |its| (|its_long|) or |foce| (|foce_long|) fitting methods. 

All these methods use a first order version of the Laplace approximation to make the computation of the likelihood function tractable. See [Wang2007]_ for details. We refer to this likelihood as the |foce| |objv|, which was first described in [Lindstrom1990]_, but is now most commonly associated with the popular |foce| fitting method in |nonmem|.

Given the same |pkpd| model, |data_file| and same |fx| and |rx| variables. |joe|, |its| and |foce| will return the same |foce| |objv|. 

The methods differ in how they update the |fx| provided in a :ref:`fit_script` by the modeller to estimate the final optimised |fx|. Given the same |fx| values all three methods obtain empirical bayes estimates of the |rx| variables in a very similar way.

.. comment
    |joe|'s |fx| update is probably most similar to |its|. 

In a |nonmem| model |foce| sometimes obtains better |fx| estimates than |its| and vice versa. Hence the existence of two fitting methods for the same |foce| |objv| in |nonmem|. In |popy| we provide |joe| as one unified approach to optimising the |foce| |objv| for a given :ref:`fit_script` and |data_file|.  

Note |joe|, |its| and |foce| fitting methods are deterministic and fundamentally different from the stochastic sampling methods such as |saem| and |imp|. |saem| and |imp| that also estimate |fx| for |pkpd| models, but the stochastic |objv_long| is |not| based on the Laplace approximation used in the |foce| |objv|.

The type of optimisation methods used by |popy| are described in references such as [DennisSchnabel1987]_ [NocedalWright2006]_.

JOE Fitting Example
---------------------

The simplest way to use |joe| in a :ref:`fit_script` is as follows:-

.. code-block:: pyml

    FIT_METHODS: [JOE:{}]

This single line, runs the |joe| fitting method with the default settings. Note that the square bracket notation '[]' is required because 'FIT_METHODS' expects a list. Or equivalently:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: {}

Where the '-' is :term:`YAML` notation for a list item.
        
One of the simplest |joe| parameters is 'max_n_main_iterations':-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100
    
Which sets the number of times the |fx| parameters are updated. The |joe| fitting proceeds iteratively with interleaved |fx| and |rx| updates. Setting this limit forces |popy| to stop after a finite number of iterations.

If you miss out 'max_n_main_iterations' it defaults to 50. Another option is as follows:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 0
            
This only updates the |rx| parameters and leaves the |fx| fixed. In |nonmem| this is achieved using the EONLY flag.

Another common setting is the 'CONVERGER' field. There are three possible settings:-

* NONE - no convergence termination - estimation stops when 'max_n_main_iterations' is reached
* OBJ_INC - terminate convergence if the |objv| increases 
* OBJ_SIM - terminate convergence if the |objv| is similar between iterations

These settings decide when a :ref:`fit_script` will return the final |fx| estimates.

The default setting for |joe| fitting is 'OBJ_INC', hence:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100
        
Is the same as:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100     
            CONVERGER: {OBJ_INC: {}}
            
And the fit estimation will stop as soon as any iteration increases the |objv|. To allow the |objv| to increase during fitting, |ie| if the likelihood gets worse between iterations, but you wish the search to continue, use:-

.. code-block:: pyml

    FIT_METHODS: 
        - JOE: 
            max_n_main_iterations: 100     
            CONVERGER: {OBJ_SIM: {}}
  
Here 'OBJ_SIM' will terminate the estimation when two consecutive iterations return similar objective values (within a relative tolerance of 1e-06).
  
Note that when fitting complex models the |objv| usually decreases at each iteration, but occasionally increases. This is due to fact that |fx| and |rx| are optimised separately. Separate optimisation is necessary for computational efficiency. However optimising components of the likelihood independently can mean the combined |objv| increases. The 'OBJ_SIM' option above is designed to allow the search to continue in these cases.
  
.. comment
    FIT_METHODS:
        - JOE:
            ODE_SOLVER:
                INHERIT: {}

            # Maximum number of iterations
            max_n_main_iterations: 30

            CONVERGER:
                OBJ_INC: {}

            # Option to recompute jacobians more often, e.g in noise optimiser.
            recompute_jacobians_flag: False

            # Option to use chain rule when computing derivatives.
            use_chain: False

            # Reset r[X]=0.0 before each r[X] optimisation
            reset_re_flag: True


.. comment

    This section defines how the model will be fitted, including the ODE (Ordinary Differential Equation) solver to be used. The user can also choose the tolerance of the ODE solver and the maximum number of iterations to use.


    The fit methods available in |popy| are:-

        - JOE:
            ODE_SOLVER:
                SCIPY_ODEINT:

                
                    # Absolute tolerance of ode solver.
                    atol: 1e-12
                    # Relative tolerance of ode solver.
                    rtol: 1e-06
                    # Maximum number of steps allowed in odeint solver.
                    max_nsteps: 750
                    # Option to use jacobian in ode solver.
                    use_jacobian: False


            # Option to output timing info.
            output_timing_flag: True
            # Maximum number of iterations
            max_n_main_iterations: 50
            # if objective function in successive main iterations varies by less than this parameter then terminate main optimisation.
            main_opt_rtol: 1e-05
            # Option to use either leastsq or sample var f[X] optimiser
            fe_var_opt_method: sample_re_ind
            # max number of fe iterations within one main iteration.
            max_n_fe_iterations: 1000
            # if objective function in successive fe iterations varies by less than this parameter then terminate fe optimisation.
            fe_opt_rtol: 0.001
            # Option to use either leastsq, leastsq_var, newton or leastsq_lmbda r[X] optimiser
            re_opt_method: leastsq
            # max number of re iterations within one main iteration.
            max_n_re_iterations: 1000
            # if objective function in successive re iterations varies by less than this parameter then terminate re optimisation.
            re_opt_rtol: 1e-06

