| Top | |
|
|
|
| NcmMPIJob * | ncm_mpi_job_ref () |
| void | ncm_mpi_job_free () |
| void | ncm_mpi_job_clear () |
| void | ncm_mpi_job_work_init () |
| void | ncm_mpi_job_work_clear () |
| NcmMPIDatatype | ncm_mpi_job_input_datatype () |
| NcmMPIDatatype | ncm_mpi_job_return_datatype () |
| gpointer | ncm_mpi_job_create_input () |
| gpointer | ncm_mpi_job_create_return () |
| void | ncm_mpi_job_destroy_input () |
| void | ncm_mpi_job_destroy_return () |
| gpointer | ncm_mpi_job_get_input_buffer () |
| gpointer | ncm_mpi_job_get_return_buffer () |
| void | ncm_mpi_job_destroy_input_buffer () |
| void | ncm_mpi_job_destroy_return_buffer () |
| gpointer | ncm_mpi_job_pack_input () |
| gpointer | ncm_mpi_job_pack_return () |
| void | ncm_mpi_job_unpack_input () |
| void | ncm_mpi_job_unpack_return () |
| void | ncm_mpi_job_run () |
| void | ncm_mpi_job_init_all_slaves () |
| void | ncm_mpi_job_run_array () |
| void | ncm_mpi_job_run_array_async () |
| void | ncm_mpi_job_free_all_slaves () |
| #define | NCM_MPI_JOB_DEBUG_PRINT() |
| typedef | NcmMPIDatatype |
| #define | NCM_TYPE_MPI_JOB |
| struct | NcmMPIJobClass |
| #define | NCM_MPI_CTRL_MASTER_ID |
| enum | NcmMPIJobCtrlMsg |
| enum | NcmMPIJobCtrlTag |
| NcmMPIJob |
GEnum ├── NcmMPIJobCtrlMsg ╰── NcmMPIJobCtrlTag GObject ╰── NcmMPIJob ├── NcmMPIJobFEval ├── NcmMPIJobFit ├── NcmMPIJobMCMC ╰── NcmMPIJobTest
This abstract class simplifies the implementation of MPI jobs through a master/slave model. Subclasses must implement virtual methods. The master dispatches jobs to the slaves, awaits results, and slaves execute the job, sending results back to the master.
For example, NcmMPIJobMCMC is a specific subclass that implements an MCMC job. Each slave creates an instance of NcmLikelihood, evaluating the likelihood function for samples received from the master.
The master/slave model leverages MPI. The subclass is responsible for implementing virtual methods for packing/unpacking input and return objects into/from MPI buffers.
When NumCosmo is compiled with MPI support, the rank 0 process executes normally,
while other ranks wait for commands from the master. The method
ncm_mpi_job_init_all_slaves() sends a serialized version of itself to all slaves.
Slaves deserialize it and wait for commands.
Each call to ncm_mpi_job_run_array() sends an array of inputs to the slaves in a
round-robin fashion. Slaves execute the job and return results to the master,
maintaining the input order. This method uses rank 0 to control the slaves, so the
master isn't involved in computations.
Conversely, ncm_mpi_job_run_array_async() sends inputs to the slaves and creates a
lightweight thread to receive results while the master concurrently executes the job.
After job completion, the master calls ncm_mpi_job_free_all_slaves() to release the
slaves. It's an error to use two or more instances of any subclass of NcmMPIJob
simultaneously.
NcmMPIJob *
ncm_mpi_job_ref (NcmMPIJob *mpi_job);
Increase the reference of mpi_job
by one.
void
ncm_mpi_job_free (NcmMPIJob *mpi_job);
Decrease the reference count of mpi_job
by one.
void
ncm_mpi_job_clear (NcmMPIJob **mpi_job);
Decrease the reference count of mpi_job
by one, and sets the pointer *mpi_job
to
NULL.
void
ncm_mpi_job_work_init (NcmMPIJob *mpi_job);
Method called after mpi_job
is initialized at the slave
and before start working.
[virtual work_init]
void
ncm_mpi_job_work_clear (NcmMPIJob *mpi_job);
Method called during the working phase of mpi_job
and in the
end before object destruction. This method can be called multiple
times during the work phase.
[virtual work_clear]
NcmMPIDatatype ncm_mpi_job_input_datatype (NcmMPIJob *mpi_job,gint *len,gint *size);
Computes the size and datatype of the input buffer.
[virtual input_datatype]
NcmMPIDatatype ncm_mpi_job_return_datatype (NcmMPIJob *mpi_job,gint *len,gint *size);
Computes the size and datatype of the return buffer.
[virtual return_datatype]
gpointer
ncm_mpi_job_create_input (NcmMPIJob *mpi_job);
Creates a new input object.
[virtual create_input]
gpointer
ncm_mpi_job_create_return (NcmMPIJob *mpi_job);
Creates a new return object.
[virtual create_return]
void ncm_mpi_job_destroy_input (NcmMPIJob *mpi_job,gpointer input);
Destroy the input
object created with ncm_mpi_job_create_input().
[virtual destroy_input]
void ncm_mpi_job_destroy_return (NcmMPIJob *mpi_job,gpointer ret);
Destroy the return
object created with ncm_mpi_job_create_return().
[virtual destroy_return]
gpointer ncm_mpi_job_get_input_buffer (NcmMPIJob *mpi_job,gpointer input);
Creates a buffer from input
compatible with ncm_mpi_job_input_datatype().
[virtual get_input_buffer]
gpointer ncm_mpi_job_get_return_buffer (NcmMPIJob *mpi_job,gpointer ret);
Creates a buffer from ret
compatible with ncm_mpi_job_return_datatype().
[virtual get_return_buffer]
void ncm_mpi_job_destroy_input_buffer (NcmMPIJob *mpi_job,gpointer input,gpointer buf);
Destroy buf
created with ncm_mpi_job_get_input_buffer()
or ncm_mpi_job_pack_input().
[virtual destroy_input_buffer]
void ncm_mpi_job_destroy_return_buffer (NcmMPIJob *mpi_job,gpointer ret,gpointer buf);
Destroy buf
created with ncm_mpi_job_get_return_buffer()
or ncm_mpi_job_pack_return().
[virtual destroy_return_buffer]
gpointer ncm_mpi_job_pack_input (NcmMPIJob *mpi_job,gpointer input);
Packs (when necessary) the input into the input buffer.
[virtual pack_input]
gpointer ncm_mpi_job_pack_return (NcmMPIJob *mpi_job,gpointer ret);
Packs (when necessary) the return into the return buffer buf
.
[virtual pack_return]
void ncm_mpi_job_unpack_input (NcmMPIJob *mpi_job,gpointer buf,gpointer input);
Unpacks (when necessary) the buffer buf
into the input pointer input
.
[virtual unpack_input]
void ncm_mpi_job_unpack_return (NcmMPIJob *mpi_job,gpointer buf,gpointer ret);
Unpacks (when necessary) the buffer buf
into the return pointer return
.
[virtual unpack_return]
void ncm_mpi_job_run (NcmMPIJob *mpi_job,gpointer input,gpointer ret);
Runs job mpi_job
using input
and returns in ret
.
[virtual run]
void ncm_mpi_job_init_all_slaves (NcmMPIJob *mpi_job,NcmSerialize *ser);
Initialize all available slaves with mpi_job
.
void ncm_mpi_job_run_array (NcmMPIJob *mpi_job,GPtrArray *input_array,GPtrArray *ret_array);
Send work to all slaves in a round-robin fashion. Both arrays input_array
and ret_array
must have the same length and should be filled with the appropriated pointers.
void ncm_mpi_job_run_array_async (NcmMPIJob *mpi_job,GPtrArray *input_array,GPtrArray *ret_array);
Send work to all slaves using an additional thread to control the slaves work.
The main execution thread runs jobs in parallel while it waits for the slaves to finish.
Both arrays input_array
and ret_array
must have the same length and should be
filled with the appropriated pointers.
void
ncm_mpi_job_free_all_slaves (NcmMPIJob *mpi_job);
Frees all available slaves used by mpi_job
.
Control messages from master to slave. All messages have the same size, specifying tag NCM_MPI_CTRL_TAG_CMD.
MPI tags for master-slave communication.
If NCM_MPI_CTRL_TAG_CMD
is used, the message is a control message.
NCM_MPI_CTRL_SLAVE_INIT must be followed by NCM_MPI_CTRL_TAG_JOB containing the serialized job.
NCM_MPI_CTRL_SLAVE_WORK must be followed by NCM_MPI_CTRL_TAG_WORK_INPUT containing the serialized input.
NCM_MPI_CTRL_SLAVE_WORK is followed by NCM_MPI_CTRL_TAG_WORK_RETURN containing the serialized return.
NCM_MPI_CTRL_SLAVE_FREE does not need to be followed by any message. The slave can be reinitialized with NCM_MPI_CTRL_SLAVE_INIT.
NCM_MPI_CTRL_SLAVE_KILL should not be followed by any message. The slave will exit.
Slaves receive messages with tags below:
Waits for a message with tag NCM_MPI_CTRL_TAG_CMD
:
If the control message is NCM_MPI_CTRL_SLAVE_INIT, the slave receives NCM_MPI_CTRL_TAG_JOB with the serialized job.
If the control message is NCM_MPI_CTRL_SLAVE_WORK, the slave receives NCM_MPI_CTRL_TAG_WORK_INPUT with the serialized input.
After NCM_MPI_CTRL_SLAVE_WORK, the slave sends NCM_MPI_CTRL_TAG_WORK_RETURN
with the serialized return.