1 Functions and macros of general purpose

1.1 Macrodefinitions to execute some mathematical operations

int dvm_mod(Op, D);

The macrodefinition returns zero, if rest from division of Op argument by D argument is equal to zero, and 1 in other case. The arguments Op and D should be integer.

<TypeOfOp> dvm_abs(Op);

The macrodefinition returns absolute value of Op argument. A type of Op argument can be integer or float. The types of returned value and Op argument are the same.

int dvm_sign(Op);

The macrodefinition returns 1, if Op is positive, and -1, if Op is zero or negative. A type of Op argument can be integer or float.

<TypeOfOp> dvm_min(Op1, Op2);

The macrodefinition returns minimal value of Op1 and Op2 arguments. A type of Op1 and Op2 arguments can be integer or float. A type of returned value is determined by C language type casting rules, applied to Op1 and Op2 arguments.

<TypeOfOp> dvm_max(Op1, Op2);

The macrodefinition returns maximal value of Op1 and Op2 arguments. A type of Op1 and Op2 arguments can be integer or float. A type of returned value is determined by C language type casting rules, applied to Op1 and Op2 arguments.

1.2 Copping arrays

1.3 High level output with processor number prefix

The functions are similarly to the C library functions printf and fprintf, but the current processor number is reported before specified information output in the following form:

ProcNumber_int (ProcNumber_ext) ,

where:

The numeration of processors assigned to the task is considered in section 4.

The functions are similarly to the functions rtl_printf and rtl_fprintf considered above, but the output is done only in the case, if internal number of the current processor is equal to parameter ProcNumber.

Note. The rtl_printf and rtl_mprintf functions are intended to output Run-Time System informational messages, which are directed usually to stdout stream (see section 8).

2 Functions and macrodefinitions of dynamic allocation of the memory

The functions and macrodefinitions, considered below, are differed from C library standard functions in the following. First, Run-Time System checks correctness of memory allocation and freeing. Secondly, allocated memory blocks can be bounded at left and at right with surplus elements, containing special code. It allows to Run-Time System during its execution to check surplus elements of the allocated blocks, that makes easy to detect errors of non-planned writing to the memory.

void *dvm_getmem (unsigned long Size);

The function allocates memory block of Size bytes and returns a pointer to the block. If the memory block of specified size cannot be allocated, user program or Run-Time System are terminated and corresponding diagnostics is output by the function eprintf (see section 7).

void *dvm_getmemnoerr (unsigned long Size);

The function is similar to the function dvm_getmem, considered above, but it returns NULL if a block is not allocated.

void *dvm_getclearmem (unsigned long Size);

The function is similar to the function dvm_getmem, considered above, but an allocated block is cleared.

The function returns to the free memory the memory block, specified by *Ptr pointer, allocates a new memory block of NewSize byte size and assigns its address to *Ptr pointer. The contents of old block will be kept in the new block (as NewSize size allows it). If new memory block of required size cannot be allocated user program or Run-Time System is terminated.

If NewSize is equal to zero, new block is not allocated and NULL is assigned to *Ptr pointer.

The function is similar to the function dvm_realloc, considered above, but it returns NULL if a block of required size is not allocated.

void dvm_freemem (void **Ptr);

The function returns to the free memory the memory block, specified by *Ptr pointer, and assigns NULL to *Ptr pointer. The function call with NULL value of *Ptr pointer is allowed.

The macrodefinition allocates memory block, containing Size elements of Type type and assigns its address to ArrayPtr. If the memory block of required size cannot be allocated user program or Run-Time System is terminated.

The macrodefinition returns to the free memory the memory block, specified by ArrayPtr pointer, allocates a new memory block of NewSize size elements of the Type type and assigns its address to ArrayPtr pointer. The contents of old block will be kept in new block (as NewSize size allows it). If new memory block of required size cannot be allocated user program or Run-Time System is terminated.

void dvm_FreeArray (void *ArrayPtr);

The macrodefinition returns to the free memory the memory block, specified by ArrayPtr pointer, and assigns NULL to ArrayPtr pointer. The macrodefinition call with NULL value of ArrayPtr pointer is allowed.

The macrodefinition allocates memory block, containing sizeof(Type) bytes and assigns its address to StructPtr pointer. If the memory block of required size cannot be allocated user program or Run-Time System is terminated.

void dvm_FreeStruct (void *StructPtr);

The macrodefinition returns to the free memory the memory block, specified by StructPtr pointer, and assigns NULL to StructPtr pointer. The macrodefinition can be invoked with NULL value of StructPtr pointer.

3 Requesting program execution time in the form, independent from hardware and software environment

double dvm_time (void);

The function returns current system (astronomical) time in seconds. The time of start measuring can differ for different hardware-software environments. The function dvm_time is implemented as C macrodefinition.

The time of program execution in seconds can be requested by the function

double dvtime_(void);

The time of start measuring returned by dvtime_ function coincides with time of user program start.

The function can be used as by Fortran program as by C program.

The time of dvtime_ function execution is more than the time of dvm_time function execution by the time of C-function call and return.

4 Requesting internal numbers of functionally provided processors

A user program startup on the processor system requires specifying (as startup parameters) the following characteristics of the processor system as multidimensional array: the processor system rank and sizes of all its dimensions. Let the rank of the processor system be n, and size of k-th dimension be PSSize_k (1 £ k £ n). Then when Run-Time System is initialized an internal number ProcNumber_int will be assigned to the each processor:

where:

So the internal number is the linear index of the processor in index space of the processor system.

Each processor has also external number ProcNumber_ext (a processor number in hardware-software environment, where Run-Time System operates). When Run-Time System is initialized, it requests the external number using environment-dependent tools.

In interprocessor exchanges a processor identifier ProcIdent is used as the processor address. The correspondence

ProcNumber_int Þ ProcIdent

is defined by Message Passing System and returned to Run-Time System when it is initialized.

There are three functionally special processors: main processor, input/output processor and central processor among processors, assigned to a task.

The main processor is the first processor, where the parallel program was initialized (a subtask, executed on the processor, is considered as initiator of all other subtasks). The main processor has zero internal number.

Input/output processor is intended to deal with the file system directly and its internal number is zero.

The central processor computes the reduction functions and is defined by an index vector ([PSSize₁/2], ... ,[PSSize_n/2]).

int rtl_GetCurrentProc(void);

The function returns internal number of the current processor.

int rtl_GetMasterProc(void);

The function returns internal number of the main processor.

int rtl_GetIOProc(void);

The function returns internal number of the input/output processor.

int rtl_GetCentralProc(void);

The function returns internal number of the central processor.

5 Message passing tools, independent from base system of message passing

A set of presented below functions serves as interface between Run-Time System itself and used message passing system (MPI, PVM and so on). This set is not belongs to the tools, provided by Run-Time System for user program. Nevertheless, minimal environment-independent set of message passing functions may be useful for test and environment-experimental programs, accompanied with Run-Time System.

The internal processor number is used as processor address in the functions, considered below. This number is substituted for processor identifier (see section 4) when the functions of message passing system are called. It is assumed that the logic of main message passing schemes (synchronous, asynchronous and “NO WAIT scheme) is known.

Memory area for sent or received message should be allocate dynamically by the functions, considered in section 2. The size of allocated memory will be more than the size of sent or received message by sizeof(double) bytes, used to keep the time of message passing function call, necessary to estimate the time of real dissynchronization (see 11.1.2). This requiment is not applied to the functions acsend_, acrecv_ (see section 5.5) and bcast_ (see section 5.4).

5.1 Synchronous message exchange

5.1.1 Synchronous sending

The function returns non-negative value in the case of success and negative value otherwise.

5.1.2 Synchronous receiving

The function returns non-negative value in the case of success and negative value otherwise.

5.2 Asynchronous message exchange

5.2.1 Asynchronous sending

The function returns non-negative value in the case of success and negative value otherwise.

5.2.2 Asynchronous receiving

The function returns non-negative value in the case of success and negative value otherwise.

5.3 Message exchange in "NO-WAIT" mode

5.3.1 Sending in "NO-WAIT" mode

The function returns non-negative value in the case of success and negative value otherwise.

5.3.2 Receiving in "NO-WAIT" mode

The function returns non-negative value in the case of success and negative value otherwise.

5.3.3 Waiting for sending or receiving completion

void rtl_Waitrequest(RTL_Request *ReqPtr);

ReqPtr - structure-flag, from which state "request has been performed" is awaited.

5.3.4 Requesting sending or receiving completion

int rtl_Testrequest(RTL_Request *ReqPtr);

ReqPtr - structure-flag, from which state "request has been performed" is awaited.

The following values are returned:

5.4 Passing message from specified processor to all other processors (broadcast passing)

The function sends the message, if internal number of the current processor is equal to SenderProcNum and receives the message otherwise.

All processors of the system, specified by PSRefPtr parameter, must belong to the current processor system. NULL value of PSRefPtr pointer or zero value of *PSRefPtr reference specify the current processor system.

The processor with SenderProcNum number may not belong to the processor system, specified by *PSRefPtr reference, but must belong to the current processor system.

Broadcast passing can be allo performed by the function

The function zeturns zero.

5.5 Message exchange between adjacent processors

The function acsend_ sends specified array to adjacent processor and the function acrecv_ reseives specified array from adjacent processor.

Adjacent (with respect to the current one) processor is defined in the following way:

Let:

Then left neighbor is the processor with coordinates C₁, … , C_k-1, … , C_n and right one is the processor with coordinates C₁, … , C_k+1, … , C_n. Left neighbor doesn't exist if C_k is zero, and right one doesn't exist if C_k is equal to the size of k-th dimension of processor system minus 1. The processor, that hasn't left or right neighbor, is named utmost one.

If m-th dimension step of parallel loop is positive acsend_ function sends specified array to right neighbor of the current processor and acrecv_ function receives specified array from left one. And vice versa if m-th dimension step is negative acsend_ function sends specified array to left neighbor of the current processor and acrecv_ function receives specified array from right one. If required for sending (or receiving) neighbor processor doesn’t exist the acsend_ (acrecv_) function does nothing (runs "idle").

The parallel loop, specified by *LoopRefPtr reference must be current and mapped. At least one dimension of the loop must be distributed.

The parameter *LoopRefPtr can be 1 or -1. It is equivalent to specifying of the reference to one-dimensional parallel loop, mapped on first dimension of current processor system. In the first case (*LoopRefPtr = 1) the loop step is positive and in the second case (*LoopRefPtr = -1) the step is negative.

The type of elements, specified by *ElmTypePtr parameter can be:

The functions return zero.

Implementation of computation according to ACROSS scheme.

At the beginning of the work all the processors performs array receiving from adjacent processor by acrecv_ function. As result all processors except one from utmost processors become in waiting state. Utmost processor (left or right depending on sign of the loop step for m-th dimension) after "idle" receiving performs required computations and then calls acsend_ function to send the array to adjacent processor, that waits receiving from utmost processor. After the array receiving and therefore waiting completion adjacent with utmost processor performs required computations and then calls acsend_ function to send the array to its neighbor. And so on. Last in this chain processor performs its computations and then performs "idle" sending, because there is no adjacent processor, required for acsend_ function.

6 Synchronization of the programs, executed at different processors

long bsynch_(void);

The function execution consists in the following: the central processor (see section 4) receives synchromessages from all other processors, and then sends synchromessages-answers to the processors. Any of the processors sends synchromessage to the central processor, and then receives synchromessage-answer from the central processor. The synchromessage is a word (integer), which content is irrelevant.

The function returns zero.

long tsynch_(void);

The function tsynch_, as well as the function bsynch_, performs barrier synchronization of the processors of the current processor system, but also aligns the time on all the processors. The time aligning means, that after execution by Run-Time System some actions the function dvm_time (see section 3) will be return identical time at all the processors at every astronomical time moment. This time is equal to the time, returned by the function dvm_time at the central processor before the function tsynch_ execution.

The time aligning is implemented in the following way. Let the central processor at the time moment t₁ (according to its own clock) sends a message to processor P, being in state of receiving this message. The processor P after receiving the message from the central processor sends to the central processor the message receiving time T (according to its own clock). The central processor after sending the message to processor P waits for the message, containing time T, from the processor, and receives it at time moment t₂ according to its own clock. Then the central processor sends time (t₁+t₂)/2 - T to the processor P. This time will be always added to the time of its clock when the function dvm_time will be executed.

The central processor performed this operation with all other processors. To avoid accidental hindrances time aligning operation is repeated several times and for every processor the value of (t₁+t₂)/2 - T, for which message passing time (t₁-t₂)/2 is minimal, is set as result.

The function returns zero.

7 Tracing user program and Run-Time System

7.1 General information about trace accumulation

Let an internal event (or event) be execution of some point or part of user program or Run-Time System under certain conditions. A system trace is accumulation of information about occurring events in their time sequence.

All calls of Run-Time System functions, accessible for users, are events. Moreover, passing through some internal parts of Run-Time System and calls of some its internal functions are also events. The examples of traced internal functions are the functions of access to communication libraries. Tracing function call caused tracing function return.

The full list of trace events is contained in the file events.def of source texts of Run-Time System. In this file a certain number is corresponded to a name of each event. The function call event has name call_<function name>, and function return event has the name ret_<function name>.

Tracing each event may be enabled or disabled by its number.

Accumulated trace information contains also all informational messages of Run-Time System and all messages about errors, detected by it (see section 8).

Run-Time System accumulates trace information, using the functions, considered in section 7.3. Each of these functions outputs information in text mode to three independently opened and closed trace streams:

• stream of Run-Time System information messages (tracing to the “screen”, usually it is stdout stream);
• memory buffer;
• file.

Tracing in informational message stream is implemented according to the general rules of the control of information message output (see section 8). Information, sent to this stream, does not contain Run-Time System informational messages and error messages.

When tracing in memory buffer the information is accumulated in a buffer of each processor up to the program completion, and then is uploaded in a file (own for each processor). Trace buffers are initialized when Run-Time System is initialized.

Two ways of tracing in a buffer are possible. In the first way the buffer overflow caused stop of tracing (the way of keeping trace beginning). In the second way last information is kept because of deleting most old information (way of “circle” tracing).

Trace in file is implemented by output of the receiving information directly in the file, own for each processor. The files for direct output of trace are opened when Run-Time System is initialized and are closed when the program is terminated.

Run-Time System places in one directory all files with accumulated trace (as files when tracing in the buffer, as files when tracing directly in files). The directory name and file extensions for each stream are specified in parameter file systrace.* (see section 8). The names of trace files have the form <internal processor number>_<external processor number>.

A structure of accumulated information is determined by trace mode. In brief mode the following information is output:

• event name;
• time, passed from previous event;
• file name of the user program and line number in the file specifying a point of event occurred in the program.

In verbose trace mode accumulated information is extended and depends on event type. If the event is a function call, then its input parameters are output. When exiting the function its result parameters are output. The verbs trace mode may be specified for each event independently on common mode.

Note. System trace is functioned only if Is_DVM_TRACE parameter kept in systrace.* files, is not equal to zero (see section 7.2.1). Nevertheless some of the functions are not traced. Such functions are the functions, requiring high execution efficiency because of supposed frequent function invoking (for example, it is the tstelm_ function of checking whether specified element of distributed array belongs to its local part or not). The functions of such kind will be traced if parameter Is_ALL_TRACE (systrace.* files) is not equal to zero. Moreover to trace functions invoking DVM-debugger Is_DEB_TRACE parameter must have non-zero value and to trace functions invoking information accumulation system for performance analysis Is_STAT_TRACE parameter must have non-zero value (see section 7.2.1).

7.2 Controlling trace accumulation by parameter files

Controlling trace modes and a set of traced events is implemented by parameters, contained in the files systrace.* and trcevent.*. Let us consider the main of them. By default, it is assumed, that a parameter is in the files systrace.*.

7.2.1 Trace enabling and disabling

• Is_DVM_TRACE - non-zero value of the parameter is the common flag of enabled tracing.
• Is_ALL_TRACE - flag to trace all Run-Time System functions (except the functions invoking DVM-debugger and statistics accumulation system).
• Is_DEB_TRACE - flag to trace the functions invoking DVM-debugger.
• Is_STAT_TRACE - flag to trace the functions invoking information accumulation system for performance analyzer.
• TraceProcList - list of internal numbers of the processors with enabled tracing. When flag of common trace is on, tracing is performed only on the processors from the list. At the main processor (see section 4) the trace information is always accumulated independently of a presence of its (zero) number in TraceProcList list. A mark of TraceProcList list end is -1.
• IsTraceProcList - flag of enabled TraceProcList list. If the parameter has zero value the trace is accumulated at all the processors.
• IsEvent[<event number>] - this parameter with value, equal to1 or 2, is the flag enabled tracing of event with the number <event number>. The value 2 specifies tracing of given event in verbose mode independently from common mode of verbose tracing. Zero value of the parameter disables specified event tracing. The parameter is in the files trcevent.*. Usually (i.e. in base sets of parameters) tracing of all events is enabled and tracing of the events with 0, 1, 2 and 3 numbers is enabled in verbose mode.
• DisableTraceEvents - list of enabled for tracing event numbers (via comma). DisableTraceEvents list disables tracing of listed events, that was enabled by IsEvent parameter. The flag of DisableTraceEvents list end is -1. Tracing of the 0, 1, 2 and 3 events can't be disabled.
• IsDisableTraceEvents - flag of enabled DisableTraceEvents list. If the parameter is zero disabling of event tracing is not performed.
• BlockTrace - flag of postponed trace enabling. When common tracing is enabled the accumulation is started only by special instruction from the function void tron(void). Enabled tracing can be disabled by the function void troff(void) (and then can be again enabled and so on).

7.2.2 Specifying opened (enabled) trace streams

• TracePath - name of directory for trace files (as files when tracing in the buffer, as files when tracing directly in files). If directory name is not empty, the last symbol of the name must be "\" or "/".
• ScreenTrace - if the parameter is not equal to zero, the trace is output to information message stream (flag to output trace to the "screen"). Trace, outputted to informational message stream, doesn’t contain informational messages and messages about errors of Run-Time System.
• BufferTrace - flag of trace accumulation in the memory buffer.
• TraceBufLength - trace buffer size in bytes.
• TraceBufferExt - trace file extension when tracing in buffer.
• FileTrace - flag to accumulate trace directly in the files.
• FatTraceNoOpen - flag to complete work in the case of trace file opening failure for direct trace accumulation. If the parameter is equal to zero, warning is output in the case of the file opening failure.
• TraceFileExt - trace file extension when tracing directly in files.

7.2.3 Trace mode specifying

• DelSysTrace - flag to delete "old" trace files (remained from previous startup) when initializing Run-Time System.
• FullTrace - common flag of verbose trace mode.
• FullTraceEvents - list of event numbers (via comma), traced in verbose mode (independently from common flag of verbose mode, specified by FullTrace parameter). The flag of FullTraceEvents list end is -1. FullTraceEvents list changes trace mode of listed events, but doesn't enable their tracing, disabled before.
• IsFullTraceEvents - flag of enabled FullTraceEvents list. If the parameter is zero trace mode of the listed in FullTraceEvents list events is not changed.
• MaxTraceLevel - common maximal traced function call enclosure. Zero value of the parameter means to trace the functions, directly called by the user program.
• MaxEventLevel[<event number>] - maximal traced function call enclosure for the event with the number <event number>. This parameter has priority under the parameter MaxTraceLevel. The parameter is in files trcevent.*.
• FullBufferStop - flag to stop tracing if buffer is overflowed. If the parameter is equal to zero, then last information is written in place of most old information when the buffer is overflowed.
• BufferTraceUnLoad – flag to upload trace, accumulated in trace buffers, into the files before program termination. If flag is equal to 0, Run-Time System does not upload trace, accumulated in trace buffers, into the files, entrusting this work to the dump-program, started if it is required.
• LowDumpLevel – flag to upload trace, accumulated in the buffers, into the files, using low-level input/output functions. Uploading trace buffers by high-level input/output functions is performed more slowly.
• SetTraceBuf - flag to trace directly into the files with buffering performed by operation system. Trace mode with buffering is set at once after opening trace stream by the function setbuf.
• TraceFlush - flag to upload trace information from operation system buffer into the file after each event (when tracing directly into the files). The mode, specified by this parameter, is active only in the case of tracing with buffering (the parameter SetTraceBuf is not equal to 0). TraceFlush parameter can have two non-zero values: 1 and 2. In the first case the trace is uploaded by fflush function. In the second case additionally trace file is closed and then opened in write mode to add information to the file end.
• DisableTraceTime – flag to subtract the execution time of trace programs from time interval between the events, which are output to the trace.

7.2.4 Controlling form of accumulated information

• KeyWordName - flag to precede the name of each event by “NAME=”.
• PreUnderLine - flag to underline heading of each event (event heading is a trace line, containing event name, time, file name and line number).
• PreUnderLining - underlining line of heading of each event.
• PostUnderLine - flag to underline end line of output information of each event.
• PostUnderLining - underlining string of the end line of information of each event.
• BufferTraceShift - coefficient of indentation of output information when tracing in buffer. The number of blanks, equal to product of this parameter by the current function call enclosure will be inserted before each output line.
• FileTraceShift - coefficient of indentation of output information when tracing directly into the files.
• CurrentTimeTrace - flag to output current system time after the time, passed from last trace event.
• TimePrecision – a number of digits after dot when printing time, passed from the previous trace event, or when printing current system time.

7.2.5 Controlling internal self-checking Run-Time System mechanisms, functioning during trace accumulation

In the process of trace information accumulation Run-Time System can check certain memory areas terminates the program in the case of such area modification and outputs corresponding error message. Such areas are:

• left (low) and right (high) control parts of each dynamically allocated memory block;
• memory areas explicitly specified by their initial and end addresses in parameter files systrace.*;
• memory area, containing program code of a user program and Run-Time System (command memory).

The control parts of dynamically allocated memory blocks are created by special instruction in parameter files sysdebug.* and are filled by the code, also specified in the files. Their contents can be checked on each memory allocation and freeing, and also in the process of tracing.

Explicitly specified memory area is checked by comparison its reference checksum with the current one. The reference checksum is calculated when Run-Time System is initialized.

The initial and end memory area addresses with program code are specified by the program, starting Run-Time System by writing these addresses to the Run-Time System executable module. The same program calculates and writes to Run-Time System executable module reference checksum of code memory. During its operating (in particularly, for trace events) it checks code memory, comparing its current checksum with reference one.

Controlling mechanisms of memory area checking is implemented by the following parameters of the files systrace.*.

• EveryEventCheckBound - flag to check all dynamically allocated memory blocks for each traced event.
• EveryTraceCheckBound - flag to check all dynamically allocated memory blocks in each call of trace output functions, described in section 7.3. The checking includes the checking specified by parameter EveryEventCheckBound, because it is performed for non-traced (disabled) events also.
• dvm_StartAddr - initial address of checked memory area when tracing.
• dvm_FinalAddr - end address of checked memory area when tracing.
• EveryEventCheckMem - flag to check the memory area, specified by parameters dvm_StartAddr and dvm_FinalAddr for each traced event.
• EveryTraceCheckMem - flag to check the memory area, specified by parameters dvm_StartAddr and dvm_FinalAddr in each call of trace output functions, described in section 7.3. The checking includes the checking specified by parameter EveryEventCheckMem, because it is performed for disabled events also.
• EveryEventCheckCodeMem - flag to check memory area with program code in each traced event.
• EveryTraceCheckCodeMem - flag to check memory area with program code in each call of trace output functions, considered in section 7.3. The checking includes the checking, specified by parameter EveryEventCheckCodeMem.

Besides specified memory area check tools, considered above, Run-Time System provides a possibility to output specified variable values to the trace. The output is controlled by the following parameters of systrace.* files:

• TraceVarAddr = <address of the variable, output to the trace>;
• TraceVarType = <type number of variable, output to the trace>;

The variable address can be decimal or hexadecimal one (according to C rules). If the address is zero, the variable is not output to the trace.

The variable type number can be:

1. int;
2. long;
3. float;
4. double;
5. char;
6. short.

If other type is specified, the variable is not output to the trace.

7.2.6 Controlling output of additional information when tracing some Run-Time System functions in extended mode

• distr_Trace – flag to output additional information when tracing the function distr_ There are reported abstract machine representation map, ranges of local values of indexes for each dimension of abstract machine representation, flag of replication for all dimensions of the processor system (Repl) and flag of mapping an element (the elements may be differ) of abstract machine representation (Every) on each processor.
• align_Trace - flag to output additional information when tracing the function align_. There are reported distributed array map and ranges of local values of indexes for each its dimension.
• dacopy_Trace - flag to output additional information when tracing functions arrcpy_ and aarrcp_. There are reported distributed array maps, ranges of local values of indexes for each dimension, information about the execution branches of the copy program and other information.
• mappl_Trace – mode of additional information output during tracing mappl_ function:

• dopl_Trace – flag to output additional information when tracing the function dopl_. The next set of ranges of local values of indexes for each dimension of the parallel loop is output.
• PLTimeTrace - flag to output the execution time information for parallel loop iteration ranges.
• RedVarTrace – flag to output addresses, types and values of all variables of reduction group when executing functions insred_, saverv_, saverd_ and strtrd_ and also at the beginning and at the end of execution of the function waitrd_. If RedVarTrace is positive, the information about first min(RedVarTrace, RedVarNumber) elements of reduction variable-array is reported.
• OutIndexTrace – flag to output result values of indexes for each dimension of distributed array when executing internal Run-Time System function GetIndexArray.
• diter_Trace – flag to output index array when the function diter_ of interconnection with DVM-debugger is traced.
• drmbuf_Trace – flag to output index array when function drmbuf_ of interconnection with DVM-debugger is traced.
• dyn_GetLocalBlock_Trace – flag to trace DVM-debugger subsidiary function dyn_GetLocalBlock.
• LoadWeightTrace - flag to trace the arrays with coordinate weights of processor loadinig when invoking setelw_ function.
• WeightArrayTrace - flag to trace the array with loading coordinate weights when exiting gettar_ function.
• AcrossTrace - flag to trace functioning ACROSS schemes.

7.3 Information output functions to trace streams

Information output to the trace streams is implemented by the following functions (the parameter format and the following parameters are similarly to correspondent parameters of printf function of C language).

int tprintf (char *format, ... );

The function outputs information to opened trace streams. Account of output symbols is returned.

The function outputs information to opened trace streams, redirected to the memory buffer and directly to the file. Moreover, the output is done to the streams for output of Run-Time System information messages (parameter prefix is equal to 0 or 1) or to the streams for error messages (prefix is equal to 2 or 3) (see section 8).

If prefix is equal to 1 or 3, the output is performed by each processor, assigned to the task, and processor number precedes to specified information (see sections 1.3 and 4). If prefix is equal to 0 or 2, each processor concerned output information to trace streams, but only main processor output information to all other streams (see section 4). The processor number is not output in this case.

A number of output symbols is returned.

The function outputs information about occurred event with the number to the opened trace streams. The information consists of:

• standard information for trace event with number;
• additional information specified by parameter format and the following parameters, if they exist.

The standard information includes:

• a name of trace event with number;
• program execution time, passed from a moment of previous actual output of information to the trace streams by the dvm_trace function;
• file name and line number, together defining a point of occurred event with number;

Actual information output to trace streams is performed only in the case, if the event with number is enabled.

The output of additional information is performed only in verbose trace mode or if the verbose mode is specified for the event with number.

The file name and line number, defining the point of occurred event, are passed to trace program by DVM_CALL macrocall. The macro has such name because of all trace events, except of several ones, are Run-Time System function calls. If the event is occurred inside Run-Time System and function call is the event, then RTL_CALL macrocall is used. The macrocalls DVM_CALL and RTL_CALL must be performed immediately before event and disposed maximally near the event in the program file. Thus event tracing is done by the following scheme:

DVM_CALL; (or RTL_CALL;)
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
< point of occurred event >
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
dvm_trace ( ........ );

Tracing Run-Time System functions is implemented in such a manner, that dvm_trace call is performed inside traced function. When tracing functions more preferable way of using DVM_CALL and DVM_TRACE macrocalls is applying operation of sequential calculation:

( DVM_CALL , < function call > )

or

( RTL_CALL , < function call > ) .

In this case line number, passed to the trace program, will be exactly equal to line number of occurred event (function call), and function call in any C expression can be equivalently changed by some of the constructions above.

Note. File name and line number, defining the point of occurred event, can be also passed to Run-Time System by the functions

Usually fname_ and dvmlf_ functions that inform Run-Time system about file name of user program, keep pointer to the file name in internal field of Run-Time system. If the length of file name (FileNameLength parameter) is positive, zero is assigned to the last byte of the file name (it is treated as a space).

While compiling Run-Time system with defined compilation variable _COPY_FILE_NAME_ the file name is copied to the internal field of the Run-Time system. If the length of the file name is less then 1 or more than 128, it will be 128. After coping zero is assigned to the last byte of kept file name.

The functions return zero values.

The functions can be used preprocessor programs to bind a point of intended event with own input text.

Note that call sequence

long line;
. . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
line = __LINE__ ;
fname_ ( __FILE__ ,
strlen(__FILE__)+1 ) ;
lnumb_ ( &line ) ;

is equivalent to macrocall

DVM_CALL ; .

The function outputs information to opened trace streams, redirected to the memory buffer and directly to a file, and to the error message streams (see section 8) and then terminates execution of a user program and Run-Time System. The current processor number is placed before the information, output to error message streams.

After output of the information, specified by parameter format and following parameters (if such ones are), the string, specified by the pointer FileNamePtr and integer, specified by parameter Line are output. It is assumed, that output string and number are file name and line number, containing given call of the function eprintf, that is eprintf call has the form:

eprintf (__FILE__ , __LINE__ , format , . . . );

Moreover, the function eprintf outputs coordinates of user program point, where the last Run-Time System call was (file name and line number).

The function is similarly to eprintf function considered above but if prefix has zero value, only main processor (see section 4) outputs information in the streams intended for error messages. The current processor number is not reported in this case.