Page Tools


    Hello, this is the right place to learn about how to deploy a mixed application with the ADAGE meta-plugin feature. This tutorial focuses on the GFarm+JuxMem interactions. You should read the corresponding documentations on GFarm and JuxMem before starting this.

    Please make sure that your JuxMem distribution has been configured with full GFarm support.

    Describing the application

    The deployment will process the following configuration:

    • A GFarm File System composed by 1 metadata server, 1 metadata cache server and 1 file system node. We add a client node, the aim of which is to prepare the GFarm environment for JuxMem.
    • A JuxMem (JXTA) overlay composed by 1 manager, 1 provider (which behaves as a GFarm client), 1 writer and 1 reader in a single Local Data Group. For further information about this configuration, please check the JuxMem basic tutorial.

    The GFarm and JuxMem sub-application descriptions are available in the adage/tests directory. We just have to create a meta-description to indicate the path to the specific descriptions, as well as the dependancies between processes of both applications. In our case, the JuxMem provider will use the GFarm configuration file set by the GFarm client. Therefore we have to make sure the GFarm client will be launched before the JuxMem provider. You can use the adage/tests/meta-gfarm-jxta-appl.xml example.

    <?xml version="1.0"?>
    <!DOCTYPE META_application>
    <META_application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="../src/plugin/meta/META_application.xsd">
     
     <specific id="gfarm-desc" path="./tests/gfarm-appl.xml"/>
     <specific id="juxmem-desc" path="./tests/jxta-appl.xml"/>
     
     <dependency deptype="acked" what="provider">
      <onwhat>gf_client_0</onwhat>
     </dependency>
     
    </META_application>

    Both provider and gf_client_0 are pg_ids you have entered in your gfarm and jxta application descriptions. The “_0” after gf_client is needed because you have to choose a specific gf_client in the case you have set multiple cardinality.

    In the control parameters file, we indicate where to find the specific control parameters file for the JXTA plugin. We also force the planner to co-locate the GFarm client with the JuxMem provider. You can use the adage/tests/meta-gfarm-jxta-ctrl-params.xml example.

    <?xml version='1.0'?>
    <!DOCTYPE ctrl_params>
    <!-- SYSTEM "../dtd/ctrl_params.dtd" -->
    <ctrl_params xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../xsd/ctrl_params.xsd">
            <submission_method name="ssh"/>
            <planner mode="random"/>
            <transfer_files method="scp"/>
            <plugin name="JXTA" file="tests/meta-gfarm-jxta-ctrl-params-spec.xml"/>
            <placement_constraints>
             <collocate id="gf_client_0" with="provider"/>
            </placement_constraints>
    </ctrl_params>

    We then enable the GFarm support in the JXTA specific control parameters file. Note that this option only makes sense in the framework of an ADAGE meta-deployment. You can use the adage/tests/meta-gfarm-jxta-ctrl-params-spec.xml example.

     <gfarm enable="true" />

    Deploying GFarm and JuxMem

    If you are not using the OAR(Grid) scheduler, this is the right time to create your resource file. Then you can type:

    ./src/adage -a tests/meta-gfarm-jxta-appl.xml -c
    tests/meta-gfarm-jxta-ctrl-params.xml -r myresourcesfile -R
    

    If you are using the OAR(Grid) scheduler you can directly indicate your reservation number(s):

    ./src/adage -a tests/meta-gfarm-jxta-appl.xml -c
    tests/meta-gfarm-jxta-ctrl-params.xml -g 10202 -g 10224 -R
    

    The deployment ends with a report indicating hosts and PIDs:

    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent37.rennes.grid5000.fr] = 4626 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent79.rennes.grid5000.fr] = 6302 6296 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent46.rennes.grid5000.fr] = 7684 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent19.rennes.grid5000.fr] = 10132 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent28.rennes.grid5000.fr] = 4643 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent28.rennes.grid5000.fr] = 4643 4669 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent91.rennes.grid5000.fr] = 25596 
    DBG[scripts_generator.cc:86] generate(): hosts_pid[paravent99.rennes.grid5000.fr] = 4507
    

    You can now monitor that your application is currently running using the generated ./get_status.sh script. You can also kill all the processes using the ./clean_up.sh script.

    You should check everything worked fine regarding the 2 applications:

    • connect to the node that hosts the JuxMem reader and check that reader.stdout contains the data expected in the JuxMem basic tutorial. By default this file is created in the local adage directory of the node ($destdir/$user/adage-$adagepid/reader/0/reader_p/0). $destdir may change depending on the node, usually it is set to /tmp.
    • connect to the node that hosts the JuxMem provider, set the GFARM_CONFIG_FILE environment variable to $destdir/$user/adage-$adagepid/share/gfarm.conf and check that JuxMem data have been flushed into the GFarm FS using gfls and gfexport.

    Powered by Heliovista - Création site internet