- last edited 03.05.01 -

Contents

1 Introduction
2 HPF features in subset HPF-DVM
3 Syntax of directives
4 DISTRIBUTE and REDISTRIBUTE directives
5 ALIGN and REALIGN directives
6 PROCESSORS directive
7 INDEPENDENT directive
8 Procedures
9 COMMON and EQUIVALENCE statements
10 Input/Output statements
References
Annex. Code examples

Example 1. Gauss elimination algorithm
Example 2. Jacobi algorithm
Example 3. Red-black successive over-relaxation

1 Introduction

This document specifies the enhanced subset of High Performance Fortran [1] (HPF-DVM).

Subset HPF-DVM inludes:

• All Fortran 77 standard [2] conforming features except for storage and sequence association (see sections 8,9).
• Subset of HPF directives (see section 2).

The HPF features included in HPF-DVM are a subset of the full HPF language chosen for their performance and their broad portability and ease of use.

Fortran 77 was selected as a base language to provide multi-machine portability, since Fortran 90 compilers are not available on all platforms.

HPF-DVM implementation is made using DVM model.

2 HPF features in subset HPF-DVM

Subset HPF-DVM is based on Fortran 77 standard and includes the following HPF1 directives.

Other HPF1 directives and language extensions are not included in the subset HPF-DVM.

3 Syntax of directives

A hpf-directive-line follows the rules of fixed form comment lines only.

4 DISTRIBUTE and REDISTRIBUTE directives

The following restrictions are imposed on DISTRIBUTE and REDISTRIBUTE directives:

• An dist-format is BLOCK or *.
• An dist-target may not be * processors-name or * (see section 8).

5 ALIGN and REALIGN directives

The following restrictions are imposed on ALIGN and REALIGN directives:

• Reversing axes, as
  CHPF$ ALIGN A(I) WITH B(100–I)
  is not permitted.
• An align-spec may not be * align-target (see section 8).
• An align-subscript-use expression must follow the syntax rules:

6 PROCESSORS directive

An explicit-shape-spec-list must be always specified in PROCESSORS directive.

Only the intrinsic function NUMBER_OF_PROCESSORS() may be used to inquire about the total number of physical processors.

7 INDEPENDENT directive

The INDEPENDENT directives may be applied only to tightly nested loops.

The iteration space of INDEPENDENT loop nest must be rectangular. That is, the lower loop bound, the upper loop bound, and the step expression for each INDEPENDENT loop must be invariant with regard to INDEPENDENT nest.

Left side of each assignment statement in INDEPENDENT nest may be only a variable with DISTRIBUTE, ALIGN, INHERIT, NEW or REDUCTION attribute.

Left sides of assignment statements of one loop iteration must be allocated at the same processor and, therefore, the loop iteration is executed on the processor entirely.

A DO-variable of INDEPENDENT loop may index distributed dimension of array only by expression of the form

a * DO-variable ± b

The integer values a and b must be invariant with regard to the INDEPENDENT loop nest.

Private (not INDEPENDENT) loops may occur inside or outside of the INDEPENDENT loop nest. A DO-variable of inside loop may index only local dimension of an array.

Syntax and semantics of REDUCTION specification correspond to HPF2 [3].

8 Procedures

1. Procedure call inside the INDEPENDENT loop.
   A procedure, called inside INDEPENDENT loop, must be free of side-effects and must not cause communication (pure procedure).
2. Procedure call outside INDEPENDENT loop.
   If the actual argument is explicitly mapped array (distributed by DISTRIBUTE or ALIGN directive), it should be passed without reshaping. It means, that actual argument is the reference to the array beginning, and configurations of actual and corresponding formal arguments are the same.
3. Formal arguments.
   If the actual argument is a distributed array, then corresponding formal argument must have inherited distribution.

9 COMMON and EQUIVALENCE statements

The arrays, distributed by default, can be used in COMMON blocks and EQUIVALENCE statements without restrictions.

The arrays, distributed by DISTRIBUTE or ALIGN directive, can't be used in EQUIVALENCE statements. Moreover, these arrays can't be associated with other data objects. Explicitly distributed arrays can be the components of COMMON block under the following conditions:

• COMMON block must be described in main program unit.
• Every occurrence of the COMMON block must have the same number of components and each corresponding component must have a storage sequences of the same size;
• If explicitly mapped array is the component of the COMMON block, then the array declarations in different program units must specify the same data type and shape. The DISRIBUTE and ALIGN directives, applied to the array, must have the identical parameters.

10 Input/Output statements

HPF-DVM allows only restricted form of input/output statements for distributed arrays:

• An input/output list may contain only one name of explicitly distributed array and may not contain other input/output items.
• Only “*” format specifier may be used in formatted I/O statement.
• A control information list may not contain ERR, END and IOSTAT specifiers.
• Any variable specified in a control information list must be variable distributed by default.

The statements of distributed array input/output cannot be used in INDEPENDENT loop.

Input/output statements for variables distributed by default have the following restrictions:

• A control information list may not contain ERR and END specifiers.
• Only the following simplified form of the implicit loop is allowed
        (A(i1,i2,...,I), I = n1,n2)
  when inputting assumed-size array distributed by default.

Input statement, INQUIRE statement, and any other input/output statement with parameter IOSTAT may not be used in an INDEPENDENT loop.

References

1. High Performance Fortran Forum. High Performance Fortran Language Specification. Version 1.0
2. ANSI X3.9-1978 Programming Language Fortran. New York 1978.
3. High Performance Fortran Forum. High Performance Fortran Language Specification. Version 2.0.

Annex. Code examples

Three small scientific programs are presented to illustrate HPF-DVM language features. They are intended for solving a system of linear equations:

A x = b

where:
A - matrix of coefficients,
b - vector of free members,
x - vector of unknowns.

The following basic methods are used for solving this system.

Direct methods. The well-known Gaussian Elimination method is the most commonly used algorithm of this class. The main idea of this algorithm is to reduce the matrix A to upper triangular form and then to use backward substitution to diagonalize the matrix.

Explicit iteration methods. Jacobi Relaxation is the most known algorithm of this class. The algorithm perform the following computation iteratively

x_i,j^new = (x_i-1,j^old + x_i,j-1^old + x_i+1,j^old + x_i,j+1^old ) / 4

Implicit iteration methods. Successive Over Relaxation (SOR) refers to this class. The algorithm performs the following calculation iteratively

x_i,j^new = ( w / 4 ) * (x_i-1,j^new + x_i,j-1^new + x_i+1,j^old + x_i,j₊₁^old ) + (1-w) * x_i,j^old

By using “red-black” coloring of variables each step of SOR consists of two half Jacobi steps. One processes “red”variables and the other processes “black” variables. The half Jacobi step calculations are data-independent.

Example 1. Gauss elimination algorithm

	PROGRAM GAUSS
C	Solving linear equation system  Ax = b
	PARAMETER  ( N = 100 )
	REAL  A( N, N+1 ), X( N )
C	A : Coefficient matrix with dimension (N,N+1)
C	Right hand side vector of linear equations is stored
C	in last (N+1)-th column of matrix A
C	X : Vector of unknowns
C	N : Number of linear equations
*HPF$	DISTRIBUTE A (BLOCK,*)
*HPF$	ALIGN X(I) WITH A(I,N+1)
C	Initialization
*HPF$	INDEPENDENT
	DO  100  I = 1, N
	DO  100  J = 1, N+1
	  IF  (( I .EQ. J )  THEN
	      A(I,J) = 2.0
	  ELSE
	    IF ( J .EQ. N+1)  THEN
	      A(I,J) = 0.0
	    ENDIF
	  ENDIF
100	CONTINUE
C
C	Elimination
C
	DO  1  I = 1, N
*HPF$	INDEPENDENT
	   DO  5  J = I+1, N
	   DO  5  K = I+1, N+1
	      A(J,K) = A(J,K) - A(J,I) * A(I,K) / A(I,I)
5	   CONTINUE
1	CONTINUE
C	First calculate X(N)
	X(N) = A(N,N+1) / A(N,N)
C
C	Solve X(N-1), X(N-2), ...,X(1) by backward substitution
C
	DO  6  J = N-1, 1, -1
*HPF$	INDEPENDENT
	   DO  7  I = 1, J
	     A(I,N+1) = A(I,N+1) - A(I,J+1) * X(J+1)
7	   CONTINUE
	   X(J) = A(J,N+1) / A(J,J)
6	CONTINUE
	PRINT *,  X
	END

Example 2. Jacobi algorithm

	PROGRAM   JACOBI
	PARAMETER  (K=8,  ITMAX=20)
	REAL  A(K,K), B(K,K)
*HPF$	DISTRIBUTE  A  (BLOCK, BLOCK)
*HPF$	ALIGN  B(I,J)  WITH  A(I,J)
C	arrays A and B  with block distribution
	PRINT *,  '********  TEST_JACOBI_HPF  ********'
C	nest of two independent loops, iteration (i,j) will be executed
C	on processor, which is owner of element A(i,j)
*HPF$	INDEPENDENT
	DO  1  J = 1, K
*HPF$	INDEPENDENT
	DO  1  I = 1, K
	   A(I,J) = 0.
	   IF(I.EQ.1 .OR. J.EQ.1 .OR. I.EQ.K .OR. J.EQ.K) THEN
		B(I,J) = 0.
	   ELSE
		B(I,J)  = 1. + I + J
	   ENDIF
1	CONTINUE
	DO  2  IT = 1, ITMAX
*HPF$	INDEPENDENT
	DO  21  J = 2, K-1
*HPF$	INDEPENDENT
	DO  21  I = 2, K-1
		A(I,J) = B(I,J)
21	CONTINUE
*HPF$	INDEPENDENT
	DO  22  J = 2, K-1
*HPF$	INDEPENDENT
	DO  22  I = 2, K-1
		B(I,J) = (A(I-1,J) + A(I,J-1) + A(I+1,J) + A(I,J+1)) / 4
22	CONTINUE
2	CONTINUE
3	OPEN (3,  FILE='JACH.DAT',  FORM='FORMATTED')
	WRITE (3,*)  B
	CLOSE (3)
	END

Example 3. Red-black successive over-relaxation

	PROGRAM REDBLACK
	PARAMETER (N1 = 20,N2 = 10)
	REAL  A(N1,N2),W
	INTEGER  ITMAX
*HPF$	DISTRIBUTE (BLOCK,BLOCK) :: A
	ITMAX = 20
	W = 0.5
*HPF$	INDEPENDENT 
	DO 1 J = 1,N2
*HPF$	  INDEPENDENT
	  DO 1 I = 1,N1
	     IF (I.EQ.J) THEN 
	        A(I,J) = N1+2
	     ELSE
	        A(I,J) = (-(1.))
	     ENDIF
1	CONTINUE
	DO 2 IT = 1,ITMAX
*HPF$	INDEPENDENT
	DO 21 J = 1,N2/2-1
*HPF$	  INDEPENDENT
	  DO 21 I = 1,N1/2-1
	     A(2*I+1,2*J+1) = W/4*(A(2*I,2*J+1)+A(2*I+2,2*J+1)+ 
     +	     A(2*I+1,2*J)+A(2*I+1,2*J+2))+(1-W)*A(2*I+1,2*J+1)
21	  CONTINUE
*HPF$	INDEPENDENT	DO 22 J = 1, N2/2-1
*HPF$	  INDEPENDENT
	  DO 22 I = 1,N1/2-1
	     A(2*I,2*J) = W/4*(A(2*I-1,2*J)+A(2*I+1,2*J)+A(2*I,2*J-1)+ 
     +	     A(2*I,2*J+1))+(1-W)*A(2*I,2*J)
22	  CONTINUE
*HPF$	INDEPENDENT
	DO 23 J = 1,N2/2-1
*HPF$	  INDEPENDENT
	  DO 23 I = 1,N1/2-1
	     A(2*I,2*J+1) = W/4*(A(2*I-1,2*J+1)+A(2*I+1,2*J+1)+A(2*I,2*J)
     +	     +A(2*I,2*J+2))+(1-W)*A(2*I,2*J+1)
23	  CONTINUE
*HPF$	INDEPENDENT
	DO 24 J = 1,N2/2-1
*HPF$	  INDEPENDENT
	  DO 24 I = 1,N1/2-1
	     A(2*I+1,2*J) = W/4*(A(2*I,2*J)+A(2*I+2,2*J)+A(2*I+1,2*J-1)+
     +	     A(2*I+1,2*J+1))+(1-W)*A(2*I+1,2*J)
24	  CONTINUE
	PRINT *,'IT= ',IT
2	CONTINUE
	END