      
! -----------------------------------------------------------------------------
! Frame   : Procedures and program used in JGR data report 
!           "Use of twenty years CLUSTER/FGM data to observe the mean behavior 
!           of the magnetic field and current density of Earth's magnetosphere" 
!           By P. Robert and M. Dunlop, October 2021
!
! Procedures for updating databases and creates 3D-grids
! Woking directory is supposed at the same level than SOFT_PRo, as DATA_PRo
!
! Author  : P. Robert, LPP-ScientiDev, 2020-Oct.2021
! Mail    : patrick.robert@lpp.polytechnique.fr
! -----------------------------------------------------------------------------

   ***************************
I- CREATE OR UPGRADE DATABASES
   ***************************

       ============================
    A- Add files to FGMPOS_database
       ============================
    
        1) download cef file
           -----------------
        
           ../SOFT_PRo/sh/RPC_download_data_oneday_CLUFGM 1 2010 10 10 SPIN
        
           => we get the ascii file C1_CP_FGM_SPIN__20101010.cef
           
           ../SOFT_PRo/sh/RPC_download_data_onemonth_CLUFGM 2020 01 SPIN
           
           => we get the ascii cef files for one month and 4 S/Sfile 
        
        2) convert cef to dat file
           -----------------------
        
           ../SOFT_PRo/bin/fgm_cef_to_dat.exe
           ->C1_CP_FGM_SPIN__20101010.cef
          
           => we get the binary file C1_FGM_SPIN_20101010.dat 
        
           This file must be moved at the right place in the FGMPOS_database
    
       ====================================
    B- Add files to FGMPOS_aligned_database
       ====================================
    
        1) prepare a list of groups of 4 FGMPOS files as following (list_to_align.txt):
           -------------------------------------------------------
        
           ....path/C1_FGM_SPIN_20040129.dat
           ....path/C2_FGM_SPIN_20040129.dat
           ....path/C3_FGM_SPIN_20040129.dat
           ....path/C4_FGM_SPIN_20040129.dat
           ....path/C1_FGM_SPIN_20040130.dat
           ....path/C2_FGM_SPIN_20040130.dat
           ....path/C3_FGM_SPIN_20040130.dat
           ....path/C4_FGM_SPIN_20040130.dat
           etc.
           
           to get all groups of FGMPOS_database do:
           
           find FGMPOS_database -name "*FGM_SPIN*.dat" |  sort -t _ -k 5 > list_dat_to_align.txt
        
           file list_dat_to_align.txt is already created in DATA_PRo
        
        2) create directories  FGMPOS_ali/2004/2004_01 (essential)
           -------------------------------------------
        
        3) product time_aligned files
           --------------------------
        
           ../SOFT_PRo/bin/alitime_fgmpos.exe < list_dat_to_align.txt
        
           => we get the binary files C*_FGM_SPIN_aligned_20040130.dat  etc.
              in the directories  FGMPOS_ali/2004/2004_01
          
           Then the produced files must be moved at the right place in the FGMPOS_aligned_database
       
       =====================================
    C- Add files to Curl_Div_database
       =====================================
    
        1) prepare a list 4 FGMPOS time_aligned files as following (list_dat_aligned.txt):
           --------------------------------------------------------------------------
        
            ./FGMPOS_ali/2010/2010_10/C1_FGM_SPIN_20101010_ali.dat
            ./FGMPOS_ali/2010/2010_10/C2_FGM_SPIN_20101010_ali.dat
            ./FGMPOS_ali/2010/2010_10/C3_FGM_SPIN_20101010_ali.dat
            ./FGMPOS_ali/2010/2010_10/C4_FGM_SPIN_20101010_ali.dat
           etc.
        
           to get all groups of FGMPOS_aligned_database do:
           
           find FGMPOS_aligned_database -name "*FGM_SPIN*.dat" |  sort -t _ -k 6 > list_dat_aligned.txt
        
           file list_dat_aligned.txt is already created in DATA_PRo
           
        2) create directories  Curl_Div_database_raw/2010/2010_10
                               Curl_Div_database-dip/2010/2010_10
                               Curl_Div_database-igrf/2010/2010_10
           (mkdir -p Curl_Div_database_raw/2010/2010_10 etc.
            or use create_Curl_Div_tree.sh))
           
           Theses directories ARE the Curl_Div databases
                               
        3) product Curl_FGM_SPIN ascii dat files
           -------------------------------------
        
           ../SOFT_PRo/bin/cocurl_from_fgmpos_ali.exe < list_aligned.txt
           
           => we get Curl_FGM_SPIN_20101010.dat files at the rigt place in the 3 directories
    

    ***************
II- CREATE 3D grids
    ***************

       ===================================================
    A- create list of all dat file from various data bases
       ===================================================
       
       find ../DATA_PRo/FGMPOS_database -name "*.dat" |  sort -t _ -k 5 > list_FGMPOS.txt
    
       find ../DATA_PRo/FGMPOS_aligned_database -name "*.dat" |  sort -t _ -k 7 > list_FGMPOS_aligned.txt
    
       find ../DATA_PRo/Curl_Div_database_raw   -name "*.dat" |  sort -t _ -k 5 > list_Curl_raw.txt
    
       find ../DATA_PRo/Curl_Div_database-dip   -name "*.dat" |  sort -t _ -k 5 > list_Curl-dip.txt
    
       find ../DATA_PRo/Curl_Div_database-igrf  -name "*.dat" |  sort -t _ -k 5 > list_Curl-igrf.txt
    
       =======================
    B- Create B and J 3D grids
       =======================
    
        1) 3D grids of B init
           ------------------
        
           rm -r grid_results #(if necessary)
           mkdir grid_results
        
          ../SOFT_PRo/bin/crea_grid_B_ini.exe < list_FGMPOS.txt
        
          mv grid_results grid_results_B_ini

          OR: crea_grid_B_ini.sh

        2) 3D grids of B init-igrf
           -----------------------
        
           rm -r grid_results #(if necessary)
           mkdir grid_results
        
          ../SOFT_PRo/bin/crea_grid_B_ini-igrf.exe < list_FGMPOS.txt
        
          mv grid_results grid_results_B_ini-igrf    

          OR: crea_grid_B_ini-igrf.sh
    
        3) 3D grids of B and J from B_raw (aligned data)
           ------------------------------
        
           rm -r grid_results #(if necessary)
           mkdir grid_results
        
          ../SOFT_PRo/bin/crea_3D_grids.exe <  list_Curl_raw.txt
        
           mv grid_results grid_results_raw

           OR: crea_3D_grids_raw.sh
        
        4) 3D grids of B and J from B-dip (aligned data)
           ------------------------------
        
           rm -r grid_results #(if necessary)
           mkdir grid_results
        
          ../SOFT_PRo/bin/crea_3D_grids.exe <  list_Curl-dip.txt
        
           mv grid_results grid_results-dip

           OR: crea_3D_grids-dip.sh
        
        5) 3D grids of B and J from B-igrf (aligned data)
           ------------------------------
        
           rm -r grid_results # (if necessary)
           mkdir grid_results
        
          ../SOFT_PRo/bin/crea_3D_grids.exe <  list_Curl-igrf.txt
        
           mv grid_results grid_results-igrf

           OR: crea_3D_grids-igrf.sh
    
       =====================================
    B- Create J 3D grids from B_ini 3D grids
       =====================================
    
        1) be sure you have the directory grid_result_B_ini
           ------------------------------------------------
        
        2) create J_from_3D_grid_Bini directory and access it:
           --------------------------------------------------
        
           mkdir J_from_3D_grid_Bini
           cd    J_from_3D_grid_Bini
        
        3) compute J from B_ini 3D grid:
           -----------------------
        
           ../../SOFT_PRo/bin/c_curl_from_B3D_grid.exe
           -> ../grid_result_B_ini/grid_Bxyz_160160160_m10.dat
        
           => we get file grid_Jxyz_160160160_m10.dat
        
           repeat process for another file (m10 => dipole tilt= -10°)
    
           OR: co_J_from_B_ini_grid.sh

       ======================================
    C- Create J 3D grids from B-igrf 3D grids
       ======================================
    
        1) be sure you have the directory grid_result_B_ini-igrf
           -----------------------------------------------------
        
        2) create J_from_3D_grid_Bini-igrf directory and access it:
           --------------------------------------------------
        
           mkdir J_from_3D_grid_Bini-igrf
           cd    J_from_3D_grid_Bini-igrf
        
        3) compute J from Bini-igrf 3D grid:
           --------------------------------
        
           ../../SOFT_PRo/bin/c_curl_from_B3D_grid.exe
           -> ../grid_result_B_ini-igrf/grid_Bxyz_160160160_m10.dat
        
           => we get file grid_Jxyz_160160160_m10.dat
        
           repeat process for another file (m10 => dipole tilt= -10°)
    
           OR: co_J_from_B_ini_grid-igrf.sh

       ============
    D- Plot 3D grid
       ============

       ../SOFT_PRo/bin/plot_3D_grid.exe
       -> answer to the questions

       OR make a files as for instance para.in containing:

       grids_result_raw_160/grid_Bxyz_160160160_m10.dat
       0 0 0
       0
       4 4 4
       0.7 3.
       0.7 3. 
       0.7 3.

       and run:

       ../SOFT_PRo/bin/plot_3D_grid.exe < para.in

       for J para.in file will be:

       ../WORK_PRo/grids_result_IGRF_80/grid_Jxyz_080080080_m10.dat
       0 0 0
       0
       5 5 5
       -1. 1.5
       -1. 1.5
       -1. 1.5
       0. 0.5
       0. 0.5
       0. 0.5
       0. 180.
       0. 180.
       0. 180.

       You can also produce all plots with:

       plot_all_B_grids.sh
       plot_all_J_grids.sh

       (very long...)

       PS plots are in the same directory as dat files.

==================================================
Any problemes: Patrick.Robert@lpp.polytechnique.fr
==================================================
    

