Fortran compiler

Troubleshooting for compilation

1. The error "Fatal: License: Unknown host." occurs.


2. The error "Invalid #line directive" occurs.


3. The error "Cannot find module: ..." occurs.


4. The error "not a valid module information file" occurs.


5. The error "Syntax error" occurs at a compiler directive.


6. The error "Invalid input for integer editing" occurs.


7. The error "Error: Invalid suffix" occurs.


8. When using a module file, a header file and a library, I want to confirm the directory to which a compiler and a linker refer.


9. The error "undefined reference to 'ftrace_region_begin_' / 'ftrace_region_end_'" occurs at linking.


10. The error "undefined reference to '__vthr$_barrier'" occurs at linking.


11. The error "undefined reference to '__vthr$_pcall_va'" occurs at linking.


12. The error "cannot find -lveproginf" and "cannot find -lveperfcnt" occurs at linking.


13. When compiling a program which code size is large, the compiler aborts by SIGSEGV.


14. The compiler aborts by SIGKILL.


15. I want to confirm whether they are executable file for VE.


16. I want to confirm whether they are object file for VE3.


17. When linking OpenMP and automatic parallelized program, which of -fopenmp and -mparallel should I specify?


18. When specifying -fcheck, compilation time becomes so long.


19. When specifying -fcheck, execution time becomes so long.


20. When specifying -ftrace, execution time becomes so long.


21. Even if setting value bigger than 8 to OMP_NUM_THREADS, threads more than 8 is not generated.


22. I want to know the name of predefined macro and the value.


23. I want to preprocess Fortran program.


24. I want to link Fortran program and C/C++ program.


25. I want to change the options of SX series to it of Vector Engine.


26. I want to change the compiler directives of SX series to it of Vector Engine.


27. The variable or routine name which name is "$" and number as "$1" is displayed in diagnostic message. What is it?


28. The type name as "DOUBLE" or "float" is displayed instead of variable name in diagnostic message. What is it?


29. The message "Internal error detected -- please report." is output.


30. The following message is output though ALLOCATE or DEALLOCATE statement is not in a loop.

    vec(181): Allocation obstructs vectorization.
    vec(182): Deallocation obstructs vectorization.
    



31. I want to know about difference between -bss and -save.


32. A compiler option which is not specified in command line is enabled.


33. I want to confirm version of the compiler.


34. I want to create a position-independent executable with the option -fpie or -fPIE.


35. The error "Too many elements in array" occurs.


36. A .L file is not generated When compiling a module source file.


37. When building a program that includes multi-stage dependencies such as a.out->foo.so->bar.so, the following link error occurs.


38. When -mparallel was specified, the following warning occurred.

Troubleshooting for execution

1. The error "Node 'N' is Offline" occurs at execution.


2. I want to confirm the used node at execution.


3. The error "./a.out: error while loading shared libraries: libnfort.so.2: cannot open shared object file: No such file or directory" is output at execution.


4. The error which a dynamic link library is not found occurs at execution.


5. The error "VE mmap failed on INTERP - TEXT: Cannot allocate memory" occurs at execution.


6. I want to confirm which line of source file corresponds to an exception occurrence point.


7. The exception occurrence point which output by traceback information is incorrect.


8. I want to output the debug write result from buffer at exception occurrence.


9. I want to confirm whether use uninitialized variable or not.


10. I want to avoid abnormal termination caused by reference of uninitialized variable.


11. A program which uses automatic parallelization and/or OpenMP is abnormally terminated by "Unable to grow stack" or SIGSEGV at execution.


12. I want to confirm how many thread was used at execution.


13. When the threads for automatic or OpenMP parallelized program execution are created or destroyed ?


14. When running a program that utilizes automatic or OpenMP parallelized, how is the number of threads determined at the runtime?


15. I want to conform the stack size required to run the program.

Troubleshooting for tuning

1. I want to confirm which optimization was applied to a program.


2. The performance decreases, though vectorization was promoted.


3. When automatic or OpenMP parallelized program is executed, the values displayed in the same item of PROGINF and FTRACE are different.


4. When using the $omp parallel num_threads (4) and executing with the environment variable OMP_NUM_THREADS =4 or OMP_NUM_THREADS=5, the execution time with OMP_NUM_THREADS=5 is a longer than with OMP_NUM_THREADS=4. Even though there are more parallel numbers.


5. The routine name which name is "$" and number as '$1' is displayed in FTRACE output. What is it?

Troubleshooting for installation

1. I want to check if the installation is correct.


2. I want to install an older version of the compiler.


3. I want to use an older version of the compiler.


4. I want to start an older version of compiler by default.


5. A message "Runtime error: vhcall_install failed." was output at execution.

Troubleshooting for SX-ACE compiler migration

1. The -ew option is specified.


2. The -A dbl option is specified.


3. The -A dbl4 option is specified.


4. The -A dbl8 option is specified.


5. The environment variable F_UFMTADJUST=TYPE2 is specified when inputting the binary file.


6. Inputting binary file created with SX-ACE.

C/C++ compiler

Troubleshooting for compilation

1. The error "Fatal: License: Unknown host." occurs.


2. The error "Syntax error" occurs at a compiler directive.


3. The error "Error: Invalid suffix" occurs.


4. When using a header file and a library, I want to confirm the directory to which a compiler and a linker refer.


5. The error "undefined reference to 'ftrace_region_begin' / 'ftrace_region_end'" occurs at linking.


6. The error "undefined reference to '__vthr$_barrier'" occurs at linking.


7. The error "undefined reference to '__vthr$_pcall_va'" occurs at linking.


8. The error "cannot find -lveproginf" and "cannot find -lveperfcnt" occurs at linking.


9. I want to confirm whether they are executable file for VE.


10. I want to confirm whether they are object file for VE3.


11. When linking OpenMP and automatic parallelized program, which of -fopenmp and -mparallel should I specify?


12. When specifying -ftrace, execution time becomes so long.


13. Even if setting value bigger than 8 to OMP_NUM_THREADS, threads more than 8 is not generated.


14. I want to know the name of predefined macro and the value.


15. The following error occur when linking C++ program.

    /opt/nec/ve/bin/nld: __curr_eh_stack_entry: TLS reference in /tmp/nccwvkaaa.o mismatches non-TLS reference in /opt/nec/ve/ncc/2.x.x/lib/libnc++.a(iostream.o)
    /opt/nec/ve/ncc/2.x.x/lib/libnc++.a: error adding symbols: Bad value
    



16. When compiling a program which code size is large, the compiler aborts by SIGSEGV.


17. The compiler aborts by SIGKILL.


18. I want to link Fortran program and C/C++ program.


19. I want to change the compiler directives of SX series to it of Vector Engine.


20. The variable or routine name which name is "$" and number as "$1" is displayed in diagnostic message. What is it?


21. The type name as "DOUBLE" or "float" is displayed instead of variable name in diagnostic message. What is it?


22. A compiler option which is not specified in command line is enabled.


23. I want to confirm version of the compiler.


24. When building a program that includes multi-stage dependencies such as a.out->foo.so->bar.so, the following link error occurs.

Troubleshooting for execution

1. The error "Node 'N' is Offline" occurs at execution.


2. I want to confirm the used node at execution.


3. The error "./a.out: error while loading shared libraries: libnfort.so.2: cannot open shared object file: No such file or directory" is output at execution.


4. The error which a dynamic link library is not found occurs at execution.


5. The error "VE mmap failed on INTERP - TEXT: Cannot allocate memory" occurs at execution.


6. I want to confirm which line of source file corresponds to an exception occurrence point.


7. The exception occurrence point which output by traceback information is incorrect.


8. I want to confirm whether use uninitialized local variable or not.


9. I want to avoid abnormal termination caused by reference of uninitialized variable.


10. A program which uses automatic parallelization and/or OpenMP is abnormally terminated by "Unable to grow stack" or SIGSEGV at execution.


11. I want to confirm how many thread was used at execution.


12. When the threads for automatic or OpenMP parallelized program execution are created or destroyed?


13. When running a program that utilizes automatic or OpenMP parallelized, how is the number of threads determined at the runtime?


14. The bus errors occur when promoting vectorization.


15. I want to conform the stack size required to run the program.

Troubleshooting for tuning

1. I want to confirm which optimization was applied to a program.


2. The performance decreases, though vectorization was promoted.


3. When automatic or OpenMP parallelized program is executed, the values displayed in the same item of PROGINF and FTRACE are different.


4. When using the $omp parallel num_threads (4) and executing with the environment variable OMP_NUM_THREADS =4 or OMP_NUM_THREADS=5, the execution time with OMP_NUM_THREADS=5 is a longer than with OMP_NUM_THREADS=4. Even though there are more parallel numbers.


5. The routine name which name is "$" and number as '$1' is displayed in FTRACE output. What is it?

Troubleshooting for installation

1. I want to check if the installation is correct.


2. I want to install an older version of the compiler.


3. I want to use an older version of the compiler.


4. I want to start an older version of compiler by default.

Fortran compiler

Troubleshooting for compilation

1. The error "Fatal: License: Unknown host." occurs.
   There is a possibility that the problem that the machine can't access a license server occurs to the time of license check of a compiler. Please refer to the FAQ indicated on a following page of HPC software license issue.
   https://www.hpc-license.nec.com/aurora/
   When not solving it, please contact us from the said page.


2. The error "Invalid #line directive" occurs.
   Directive of preprocessors such as "#if, #include" is used. Please compile with -fpp.


3. The error "Cannot find module: ..." occurs.
   A module was used, but the compiler could not find the module file (*.mod). Please confirm whether a module file exists in the directory by which a compiler searches a module file. Please refer to "Fortran Compiler User's Guide" about the directory a compiler searches.


4. The error "not a valid module information file" occurs.
   There is a possibility that a module file was compiled by an old compiler or is broken. Please remake a module file (*.mod).


5. The error "Syntax error" occurs at a compiler directive.
   Please confirm whether the spelling of compiler directive and the how to use aren't wrong. When it's an error to compiler directive of a SX compiler, please change to it of a VE compiler by a compiler directive line change tool. Please refer to "Fortran Compiler User's Guide" to confirm the usage of the tool.


6. The error "Invalid input for integer editing" occurs.
   This error occurs when an integer type input data of list-directed input or NAMELIST input is incorrect.
   This error can occur when the following cases. Please check input data in an input file.
   
   
   • Example: The character following the sign (+-) is not a number.

     -  10
     

     This error occurs due to a space between the sign and the number.
   
   
   • Example: The previous data is in R*C format, the data specified by R still remains, and the data of C is not an integer type.

     10*1.0
     

     When the input data in the above example is read to an array such as real::a(9), 9 out of 10 data are used as the value of array "a", but one input data remains. And then, if input to integer variable is executed, the remaining one data (real type) is used as input data to the integer variable, and this error occurs.
   
   
   • Example: The character following the integer data is not a delimiter (space, tab, line break, comma diagonal, &, $).

     20!
     10*20!
     

     This error occurs due to a number followed by a non-delimiter.


7. The error "Error: Invalid suffix" occurs.
   There is a possibility that binutils-ve package is old. Please confirm whether binutils-ve package is the latest edition.


8. When using a module file, a header file and a library, I want to confirm the directory to which a compiler and a linker refer.
   Please refer to "Fortran Compiler User's Guide".


9. The error "undefined reference to 'ftrace_region_begin_' / 'ftrace_region_end_'" occurs at linking.
   The FTRACE function is used. Specify -ftrace at linking. Please refer to "PROGINF/FTRACE User's guide" about the FTRACE function.
   
   

   $ nfort a.o b.o -ftrace
   



10. The error "undefined reference to '__vthr$_barrier'" occurs at linking.
    Please specify -mparallel or -fopenmp at linking.


11. The error "undefined reference to '__vthr$_pcall_va'" occurs at linking.
    Please specify -mparallel or -fopenmp at linking.


12. The error "cannot find -lveproginf" and "cannot find -lveperfcnt" occurs at linking.
    Please install nec-veperf package.


13. When compiling a program which code size is large, the compiler aborts by SIGSEGV.
    The stack size needed by the compiler may exceed upper limit of the setting. It may solve to extend the upper limit of it. It can be confirm and setting to invoke "ulimit -s" as follows. Please increase the upper limit of stack size and recompile the program.
    
    

    $ ulimit -s 			(Check the current limit)
    8192
    $ ulimit -s 16384		(Change the limit)
    



14. The compiler aborts by SIGKILL.
    The memory of the machine may be exhausted. The memory used amount can be somewhat reduced to compile with -O0 or -O1.


15. I want to confirm whether they are executable file for VE.
    Please execute "/opt/nec /ve/bin/nreadelf -h" that specified the executable file as an argument of command. When "NEC VE architecture" is output in the line of "Machine:", it show that a file is an executable file for VE.
    
    

    $ /opt/nec/ve/bin/nreadelf -h a.out
    ELF Header:
      Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
      Class:                             ELF64
      Data:                              2's complement, little endian
      Version:                           1 (current)
      OS/ABI:                            UNIX - System V
      ABI Version:                       0
      Type:                              EXEC (Executable file)
      Machine:                           NEC VE architecture
      (...)
    



16. I want to confirm whether they are object file for VE3.
    Please execute "/opt/nec/ve/bin/nreadelf -h" that specified the object file as an argument of command. When the last digit output in the line of "Flags:" is "0", it show that a file is an object file for VE1; when it is "1", it show that a file is an object file for VE3. In the following example, the last digit output in the line of "Flags:" is "1", so it show that a file is an object file for VE3.
    
    

    $ /opt/nec/ve/bin/nreadelf -h a.o
    ELF Header:
      (...)
      Version:                           0x1
      Start of program headers:          0 (bytes into file)
      Start of section headers:          720 (bytes into file)
      Flags:                             0x10101
      (...)
    



17. When linking OpenMP and automatic parallelized program, which of -fopenmp and -mparallel should I specify?
    Please specify either -fopenmp or -mparallel.

    $ nfort -c -mparallel a.f90
    $ nfort -c -fopenmp b.f90
    $ nfort -fopenmp a.o b.o
    



18. When specifying -fcheck, compilation time becomes so long.
    It becomes long because check code is inserted at compilation. Please specify -fcheck to only the source file which includes procedure which need check.


19. When specifying -fcheck, execution time becomes so long.
    It becomes long because check code is executed. Please specify -fcheck to only the source file which includes procedure which need check.


20. When specifying -ftrace, execution time becomes so long.
    It becomes long because extra routines for getting performance information are executed at entrance/exit of procedures and user specified regions. Please specify -ftrace to only the source file which includes routine which performance information is required.


21. Even if setting value bigger than 8 to OMP_NUM_THREADS, threads more than 8 is not generated.
    8 threads are the upper limit because the number of cores of VE is 8.


22. I want to know the name of predefined macro and the value.
    Please refer to "Fortran Compiler User's Guide".


23. I want to preprocess Fortran program.
    Please compile the program with -fpp.


24. I want to link Fortran program and C/C++ program.
    Please refer to description of Language-Mixed Programming in "Fortran Compiler User's Guide".


25. I want to change the options of SX series to it of Vector Engine.
    Please change it to refer to the correspondence tables of compiler options between SX compilers and compilers for the Vector Engine in "Fortran Compiler User's Guide".


26. I want to change the compiler directives of SX series to it of Vector Engine.
    Please use the "Compiler Directive Conversion Tool" or change by hand by confirming the correspondence tables of compiler directives between SX compilers and compilers for the Vector Engine in "Fortran Compiler User's Guide". Please refer to "Fortran Compiler User's Guide" about the tool.


27. The variable or routine name which name is "$" and number as "$1" is displayed in diagnostic message. What is it?
    It is created by compiler to do vectorization and parallelization.


28. The type name as "DOUBLE" or "float" is displayed instead of variable name in diagnostic message. What is it?
    It is unnamed variable created by compiler to do vectorization and parallelization. It is displayed type name because it has no name.


29. The message "Internal error detected -- please report." is output.
    When compilation is not stopped at the message output, the compiler recover the error and continues compiling. In this case, created object file can be used without problems.
    When compilation is stopped, please contact us from the NEC support portal site.


30. The following message is output though ALLOCATE or DEALLOCATE statement is not in a loop.

    vec(181): Allocation obstructs vectorization.
    vec(182): Deallocation obstructs vectorization.
    

    
    This message is output when the compiler needed to allocate and deallocate an area at execution to realize language specification of Fortran. It may occur when passing argument or return value at inlining a procedure.


31. I want to know about difference between -bss and -save.
    In case of variable of SAVE attribute, initialized value in a routine is return value of called last time. In case of -bss, it is not guaranteed.


32. A compiler option which is not specified in command line is enabled.
    A compiler option may be specified in option file. Please refer to "Fortran Compiler User's Guide" to confirm details of option file.


33. I want to confirm version of the compiler.
    Please compile with --version.


34. I want to create a position-independent executable with the option -fpie or -fPIE.
    Creation of a position-independent executable is not supported.


35. The error "Too many elements in array" occurs.
    The size of the array allocated by the ALLOCATE statement, or the size of the array allocated by the DIMENSION statement/attribute, exceeds 1TiB. Please review the size of array.
    Note: The upper limit of the array size is checked with 1TiB at compilation, but the memory size of VE is 48GB. Therefore, if you try to allocate the array larger than the memory size of VE, it occurs "Out of memory" at run-time.


36. A .L file is not generated When compiling a module source file.
    A L files is not generated for module source files that do not contain module procedures according to its specifications.


37. When building a program that includes multi-stage dependencies such as a.out->foo.so->bar.so, the following link error occurs.
    

    /opt/nec/ve/bin/nld: warning: libbar.so, needed by ./libfoo.so, not found
    (try using -rpath or -rpath-link)
    ./libfoo.so: undefined reference to `bar'
    

    
    It is a GNU Linker specification from which nld is derived. The nld links SX-Aurora TSUBASA objects on Linux/x86_64, so it works with cross linker. Since Cross Linker is not always the same as the actual execution environment, nld ignores the -rpath option and RPATH set in the shared library. Please specify -Wl,-rpath-link,library-path.


38. When -mparallel was specified, the following warning occurred.
    

    /opt/nec/ve/bin/nld: warning: libnfort.so.2, needed by libxxx.so, not found (try using -rpath or -rpath-link)
    

    
    libnfort.so is a library that contains Fortran runtime routines and is required whenever linking a non-parallel version of a Fortran program. When linking a parallel version of a Fortran program (with -mparallel or -fopenmp), libnfort_m.so is always required instead. The compiler automatically specifies "-lnfort" for non-parallel and "-lnfort_m" for parallel at the time of linking. The warning is that libxxx.so is created non-parallel and requires a non-parallel libnfort.so, but "-lnfort" is required because "-lnfort" is not specified when -mparallel is specified.
    Since unexpected problems may occur when running the libxxx.so, it is recommended to additionally specify -mparallel or -fopenmp when creating (linking) the libxxx.so to reference libnfort_m.so.

Troubleshooting for execution

1. The error "Node 'N' is Offline" occurs at execution.
   The state of VE node of number N is OFFLINE. Please make it ONLINE.
   The example which make VE node of number 0 ONLINE state is as follows.

   # /opt/nec/ve/bin/vecmd -N 0 state set on
   ...
   Result: Success
   # /opt/nec/ve/bin/vecmd state get
   ...
   ----------------------------------------------------------------
   VE0 [03:00.0] [ ONLINE ] Last Modif:2017/11/29 10:18:00
   ----------------------------------------------------------------
   Result: Success
   



2. I want to confirm the used node at execution.
   Please execute the command /opt/nec /ve/bin/ps. The command ps outputs snapshot of executing processes by VE node. In the following example, it can be confirmed that the program named "a.out" is executing on VE node of number 2.

   /opt/nec/ve/bin/ps -a
   VE Node: 3
     PID TTY          TIME CMD
   VE Node: 1
     PID TTY          TIME CMD
   VE Node: 2
     PID TTY          TIME CMD
   50727 pts/1    00:01:36 a.out
   VE Node: 0
     PID TTY          TIME CMD
   



3. The error "./a.out: error while loading shared libraries: libnfort.so.2: cannot open shared object file: No such file or directory" is output at execution.
   Please install the package "nec-nfort-shared" and "nec-nfort-shared-inst". Please follow the instructions described in the "Installation Guide".


4. The error which a dynamic link library is not found occurs at execution.
   Please set the directory which dynamic link library is put to the environment variable VE_LD_LIBRARY_PATH. Please refer to "Fortran Compiler User's Guide".


5. The error "VE mmap failed on INTERP - TEXT: Cannot allocate memory" occurs at execution.
   The total size of text and static data of the VE program exceeds 48GB. Please modify the source files to dynamically allocate large-sized data so that the total size of the text and static data of the VE program is 48GB or less. Static data includes module variables, block data, common blocks, and variables with SAVE attribute. You can check the symbol sizes in descending order by executing the following command for reference:

   $ /opt/nec/ve/bin/nnm -C --size-sort -r ./a.out
   

   In addition, you can check the size of each section with the following command:

   $ /opt/nec/ve/bin/nreadelf -e ./a.out
   



6. I want to confirm which line of source file corresponds to an exception occurrence point.
   It can be check by traceback information. Please refer to "Fortran Compiler User's Guide" to check process of it.


7. The exception occurrence point which output by traceback information is incorrect.
   The exception occurrence point output by traceback information can be incorrect by the advance control of HW. The advance control can be stopped to set the environment variable VE_ADVANCEOFF=YES. An execution time may increase substantially to stop the advance control. Please take care it.

   $ export VE_ADVANCEOFF=YES
   



8. I want to output the debug write result from buffer at exception occurrence.
   Please call the FLUSH statement after the WRITE statement.

   SUBROUTINE SUB()
     INTEGER :: U, X
   
     OPEN(NEWUNIT=U, FILE='debug.log', STATUS='replace')
   
     CALL SUB1(X)
   
   #ifdef DEBUG
     WRITE(U, *) 'X=', X
     FLUSH(U)
   #endif
   
   END
   



9. I want to confirm whether use uninitialized variable or not.
   It may be checked by detecting an exception to compile with -minit-stack=snan and execute with the environment variable VE_INIT_HEAP=SNAN for double precision floating-point type variables. For single precision floating-point type variables, specify snanf and SNANF instead of snan and SNAN. This approach can be used only if the variable is floating-point type.


10. I want to avoid abnormal termination caused by reference of uninitialized variable.
    It may avoid by initializing the area to zero to compile with -minit-stack=zero and execute with the environment variable VE_INIT_HEAP=ZERO. Correction of a program is recommended to resolve a potential problem.


11. A program which uses automatic parallelization and/or OpenMP is abnormally terminated by "Unable to grow stack" or SIGSEGV at execution.
    It may occur because the amount of stack usage exceeds the limit. Please increase the limit of stack size or decrease the stack usage.
    1. The limit of stack size can be increased by setting the environment variable OMP_STACKSIZE.

       $ export OMP_STACKSIZE=2G
       

    2. The used stack can be decreased to specify the -mno-stack-arrays. Please note that the execution time can be increased by specifying .mno-stack-arrays.


12. I want to confirm how many thread was used at execution.
    It can be confirmed to check "Max Active Threads" in PROGINF. "Max Active Threads" is output to stderr at termination when setting the environment variable "VE_PROGINF=DETAIL". Please refer to "PROGINF/FTRACE user's Guide" to confirm usage of PROGINF.
    In the following example, it can be confirmed that 4 thread was used because "Max Active Threads" is 4.

    ******** Program Information ********
    (...)
    Power Throttling (sec)			:		0.000000
    Thermal Throttling (sec)		:		0.000000
    Max Active Threads			:		4
    Available CPU Cores			:		8
    Average CPU Cores Used			:		3.323850
    Memory Size Used (MB)			:		7884.000000
    Start Time (date)			:		Mon Feb 19 04:43:34 2018 JST
    End Time (date)				:		Mon Feb 19 04:44:08 2018 JST
    



13. When the threads for automatic or OpenMP parallelized program execution are created or destroyed?
    By default, the threads are created at the start of execution and destroyed at termination. The number of threads are the specified value by the environment variable OMP_NUM_THREADS or VE_OMP_NUM_THREADS. If it is not specified, the number is the same as the number of available VE cores.


14. When running a program that utilizes automatic or OpenMP parallelized, how is the number of threads determined at the runtime?
    The number of threads at runtime can be specified through the environment variables OMP_NUM_THREADS or VE_OMP_NUM_THREADS, the OpenMP num_threads clause, and the omp_set_num_threads() function. The priority is as follows:
    1. Value specified by num_threads clause
    2. Value specified by omp_set_num_threads() function
    3. Value specified by the environment variable VE_OMP_NUM_THREADS
    4. Value specified by the environment variable OMP_NUM_THREADS
    5. The same value as the number of available VE cores.
    The number of threads at execution is the same as the number of available VE cores if it is set a value greater than the number of available VE cores in num_threads clause, omp_set_num_threads(), VE_OMP_NUM_THREADS, or OMP_NUM_THREADS.


15. I want to conform the stack size required to run the program.
    There is no way to find out the required stack size because you will not know it until you try it.
    In the case of Fortran programs, 192 MB of memory is pre-allocated at the start of the program as a region to speed up memory allocation at execution, such as for ALLOCATE statements. Please note that this allocation occurs regardless of whether ALLOCATE statements are actually used or not.

Troubleshooting for tuning

1. I want to confirm which optimization was applied to a program.
   Please refer to output diagnostics and the format list when compiling. The diagnostics list is output when the compiler option -report-diagnostics, and the format list is output when the compiler option -report-format is specified.


2. The performance decreases, though vectorization was promoted.
   The performance decreases by an overhead of vectorization of the few iteration loop. Please specify the novector directive to such loop to stop vectorization.


3. When automatic or OpenMP parallelized program is executed, the values displayed in the same item of PROGINF and FTRACE are different.
   The number of operations for the spin-waiting of the thread created before main program starts is added in PROGINF, but not in FTRACE.


4. When using the $omp parallel num_threads (4) and executing with the environment variable OMP_NUM_THREADS =4 or OMP_NUM_THREADS=5, the execution time with OMP_NUM_THREADS=5 is a longer than with OMP_NUM_THREADS=4. Even though there are more parallel numbers.
   When the value passed with the num_threads clause is different from the value specified with the environment variable OMP_NUM_THREADS, the execution time increases due to thread regeneration.
   Threads are automatically generated before the main program starts. The number of threads is determined by the environment variable OMP_NUM_THREADS. When the number of threads changes in the program with the function omp_set_thread_num() or num_threads clause in OpenMP, the threads generated before the main program starts is freed and the new threads are regenerated.


5. The routine name which name is "$" and number as '$1' is displayed in FTRACE output. What is it?
   It is created by compiler to do vectorization and parallelization.

Troubleshooting for installation

1. I want to check if the installation is correct.
   Please specify the --version option to check the version. If the displayed version number is the same as the installed property, it has been installed correctly. The version number is output to X.X.X in the following example.

   $ /opt/nec/ve/bin/nfort --version
   nfort (NFORT) X.X.X (Build 14:10:47 Apr 23 2020)
   Copyright (C) 2018,2020 NEC Corporation.
   



2. I want to install an older version of the compiler.
   Please refer to "A.1.1 Installation of a Specific Version of the Compilers" in the SX-Aurora TSUBASA Installation Guide to install old versions of the compiler.


3. I want to use an older version of the compiler.
   Please invoke /opt/nec /ve/bin/nfort-X.X.X, ncc-X.X.X, or nc++-X.X.X (X.X.X is the version number of the compiler) at compilation. For details, refer to "Fortran Compiler User's Guide".


4. I want to start an older version of compiler by default.
   There are two ways to do it. Please choose one.
   
   1. The substance of each version of ncc /nc++/nfort commands are installed as follows. X.X.X is the version number of the compiler.

      /opt/nec/ve/ncc/X.X.X/bin/ncc
      /opt/nec/ve/ncc/X.X.X/bin/nc++
      /opt/nec/ve/nfort/X.X.X/bin/nfort
      

      Set the bin directory of the version you want to invoke by default to the command search path (environment variable PATH).
   
   
   2. Installing an older version of compiler.
   Install the package of the compiler version you want to set as default. For details, refer to "A.1.2 Change of the Compiler Versions Invoked with the Command /opt/nec/ve/bin/[nfort|ncc|nc++]" in the SX-Aurora TSUBASA Installation Guide. Please note that this method will affect all users who use /opt/nec/ve/bin/nfort (ncc, nc++).


5. A message "Runtime error: vhcall_install failed." was output at execution.
   The library needed to offload array lumped input/output, byte swap processing, etc., to VH is either missing or not installed.
   Please try the following:
   
   • Add the directory name "/opt/nec/ve/nfort/lib64" to the environment variables VE_LD_LIBRARY_PATH / LD_LIBRARY_PATH.
   • If the above does not resolve the issue, check if the file "libnfort_x86_64.so.(version number)" exists in the directory "/opt/nec/ve/nfort/lib64". If it's missing, download and install "nec-nfort-runtime-(version number)-1.x86_64.rpm" from the yum repository.

Troubleshooting for SX-ACE compiler migration

1. The -ew option is specified.
   Check the program to see if it applies to the following:
   1. When you are using intrinsic procedures by specific-name, modify it to a doubleprecision or generic-name.
   2. Modify the type declarations and constants in the program as shown in the following.

   Fortran90/SX Compiler	Vector Engine Compiler
   INTEGER*2	INTEGER*8
   INTEGER*4	INTEGER*8
   INTEGER(KIND=2)	INTEGER(KIND=8)
   INTEGER(KIND=4)	INTEGER(KIND=8)
   LOGICAL*1	LOGICAL*8
   LOGICAL*4	LOGICAL*8
   LOGICAL(KIND=1)	LOGICAL(KIND=8)
   LOGICAL(KIND=4)	LOGICAL(KIND=8)
   REAL*4	REAL*8
   REAL(KIND=4)	REAL(KIND=8)
   COMPLEX*8	COMPLEX*16
   COMPLEX(KIND=4)	COMPLEX(KIND=8)
   Constants 1.23E1	Constants 1.23D1
   Constants 1.23_4	Constants 1.23_8

   3. Specify both options -fdefault-real=8 and -fdefault-integer=8 when compiling. This compiler option is not required when you modified program to specify the kind type in a type declaration.


2. The -A dbl option is specified.
   Please do one of the following.
   1. Modify the type declarations and constants in the program as shown in the following.

   Fortran90/SX Compiler	Vector Engine Compiler
   REAL*4	REAL*8
   REAL*8	REAL*16
   REAL(KIND=4)	REAL(KIND=8)
   REAL(KIND=8)	REAL(KIND=16)
   COMPLEX*8	COMPLEX*16
   COMPLEX*16	COMPLEX*32
   COMPLEX(KIND=4)	COMPLEX(KIND=8)
   COMPLEX(KIND=8)	COMPLEX(KIND=16)
   Constants 1.23E1	Constants 1.23D1
   Constants 1.23D1	Constants 1.23Q1
   Constants 1.23_4	Constants 1.23_8
   Constants 1.23_8	Constants 1.23_16

   2. Specify both options -fdefault-real=8 and -fdefault-double=16 when compiling. This compiler option is not required when you modified program to specify the kind type in a type declaration.


3. The -A dbl4 option is specified.
   Please do one of the following.
   1. Modify the type declarations and constants in the program as shown in the following.

   Fortran90/SX Compiler	Vector Engine Compiler
   REAL*4	REAL*8
   REAL(KIND=4)	REAL(KIND=8)
   COMPLEX*8	COMPLEX*16
   COMPLEX(KIND=4)	COMPLEX(KIND=8)
   Constants 1.23E1	Constants 1.23D1
   Constants 1.23_4	Constants 1.23_8

   2. Specify options -fdefault-real=8 when compiling. This compiler option is not required when you modified program to specify the kind type in a type declaration.


4. The -A dbl8 option is specified.
   Please do one of the following.
   
   • 1. Modify the type declarations and constants in the program as shown in the following.

     Fortran90/SX Compiler	Vector Engine Compiler
     REAL*8	REAL*16
     REAL(KIND=8)	REAL(KIND=16)
     COMPLEX*16	COMPLEX*32
     COMPLEX(KIND=8)	COMPLEX(KIND=16)
     Constants 1.23D1	Constants 1.23Q1
     Constants 1.23_8	Constants 1.23_16

     2. Specify option -fdefault-double=16 when compiling. This compiler option is not required when you modified program to specify the kind type in a type declaration.
   
   
   • The environment variable F_UFMTADJUST=TYPE2 is specified when inputting the binary file.
     Specify the environment variable VE_FORT_UFMTADJUST, when inputting binary file that specified and created by the environment variable F_UFMTADJUST.
   
   
   • Inputting binary file created with SX-ACE.
     Specify the environment variable VE_FORT_UFMTENDIAN, when inputting binary file created with SX-ACE.
   
   

C/C++ compiler

Troubleshooting for compilation

1. The error "Fatal: License: Unknown host." occurs.
   There is a possibility that the problem that the machine can't access a license server occurs to the time of license check of a compiler. Please refer to the FAQ indicated on a following page of HPC software license issue.
   https://www.hpc-license.nec.com/aurora/
   When not solving it, please contact us from the said page.


2. The error "Syntax error" occurs at a compiler directive.
   The error "Syntax error" occurs at a compiler directive. Please confirm whether the spelling of compiler directive and the how to use aren't wrong. When it's an error to compiler directive of a SX compiler, please change to it of a VE compiler by a compiler directive line change tool. Please refer to "C/C++ Compiler User's Guide" to confirm the usage of the tool.


3. The error "Error: Invalid suffix" occurs.
   There is a possibility that binutils-ve package is old. Please confirm whether binutils-ve package is the latest edition.


4. When using a header file and a library, I want to confirm the directory to which a compiler and a linker refer.
   Please refer to "C/C++ Compiler User's Guide".


5. The error "undefined reference to 'ftrace_region_begin' / 'ftrace_region_end'" occurs at linking.
   The FTRACE function is used. Specify -ftrace at linking. Please refer to "PROGINF/FTRACE User's guide" about the FTRACE function.

   $ ncc a.o b.o -ftrace
   



6. The error "undefined reference to '__vthr$_barrier'" occurs at linking.
   Please specify -mparallel or -fopenmp at linking.


7. The error "undefined reference to '__vthr$_pcall_va'" occurs at linking.
   Please specify -mparallel or -fopenmp at linking.


8. The error "cannot find -lveproginf" and "cannot find -lveperfcnt" occurs at linking.
   Please install nec-veperf package.


9. I want to confirm whether they are executable file for VE.
   Please execute "/opt/nec /ve/bin/nreadelf -h" that specified the executable file as an argument of command. When "NEC VE architecture" is output in the line of "Machine:", it show that a file is an executable file for VE.

   $ /opt/nec/ve/bin/nreadelf -h a.out
   ELF Header:
     Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
     Class:                             ELF64
     Data:                              2's complement, little endian
     Version:                           1 (current)
     OS/ABI:                            UNIX - System V
     ABI Version:                       0
     Type:                              EXEC (Executable file)
     Machine:                           NEC VE architecture
     (...)
   



10. I want to confirm whether they are object file for VE3.
    Please execute "/opt/nec/ve/bin/nreadelf -h" that specified the object file as an argument of command. When the last digit output in the line of "Flags:" is "0", it show that a file is an object file for VE1; when it is "1", it show that a file is an object file for VE3. In the following example, the last digit output in the line of "Flags:" is "1", so it show that a file is an object file for VE3.
    
    

    $ /opt/nec/ve/bin/nreadelf -h a.o
    ELF Header:
      (...)
      Version:                           0x1
      Start of program headers:          0 (bytes into file)
      Start of section headers:          720 (bytes into file)
      Flags:                             0x10101
      (...)
    



11. When linking OpenMP and automatic parallelized program, which of -fopenmp and -mparallel should I specify?
    Please specify either -fopenmp or -mparallel.

    $ ncc -c -mparallel a.c
    $ ncc -c -fopenmp b.c
    $ ncc -fopenmp a.o b.o
    



12. When specifying -ftrace, execution time becomes so long.
    It becomes long because the routine for getting performance information is executed. It is called at entrance/exit of functions and user specified region.
    Please specify -ftrace to only the source file which includes routine which performance information is required.


13. Even if setting value bigger than 8 to OMP_NUM_THREADS, threads more than 8 is not generated.
    8 threads are the upper limit because the number of cores of VE is 8.


14. I want to know the name of predefined macro and the value.
    Please refer to "C/C++ Compiler User's Guide".


15. The following error occur when linking C++ program.

    /opt/nec/ve/bin/nld: __curr_eh_stack_entry: TLS reference in /tmp/nccwvkaaa.o mismatches non-TLS reference in /opt/nec/ve/ncc/2.x.x/lib/libnc++.a(iostream.o)
    /opt/nec/ve/ncc/2.x.x/lib/libnc++.a: error adding symbols: Bad value
    

    
    Please recompile the program by the compiler of version 2.2.1 or later.


16. When compiling a program which code size is large, the compiler aborts by SIGSEGV.
    The stack size needed by the compiler may exceed upper limit of the setting. It may solve to extend the upper limit of it. It can be confirm and setting to invoke "ulimit -s" as follows. Please increase the upper limit of stack size and recompile the program.

    $ ulimit -s 			(Check the current limit)
    8192
    $ ulimit -s 16384		(Change the limit)
    



17. The compiler aborts by SIGKILL.
    The memory of the machine may be exhausted. The memory used amount can be somewhat reduced to compile with -O0 or -O1.


18. I want to link Fortran program and C/C++ program.
    Please refer to "C/C++ Compiler User's Guide".


19. I want to change the compiler directives of SX series to it of Vector Engine.
    Please use the "Compiler Directive Conversion Tool" or change by hand by confirming the correspondence tables of compiler directives between SX compilers and compilers for the Vector Engine in "Fortran Compiler User's Guide".


20. The variable or routine name which name is "$" and number as "$1" is displayed in diagnostic message. What is it?
    It is created by compiler to do vectorization and parallelization.


21. The type name as "DOUBLE" or "float" is displayed instead of variable name in diagnostic message. What is it?
    It is unnamed variable created by compiler to do vectorization and parallelization. It is displayed type name because it has no name.


22. A compiler option which is not specified in command line is enabled.
    A compiler option may be specified in option file. Please refer to "C/C++ Compiler User's Guide" to confirm details of option file.


23. I want to confirm version of the compiler.
    Please compile with --version.


24. When building a program that includes multi-stage dependencies such as a.out->foo.so->bar.so, the following link error occurs.
    

    /opt/nec/ve/bin/nld: warning: libbar.so, needed by ./libfoo.so, not found
    (try using -rpath or -rpath-link)
    ./libfoo.so: undefined reference to `bar'
    

    It is a GNU Linker specification from which nld is derived. The nld links SX-Aurora TSUBASA objects on Linux/x86_64, so it works with cross linker. Since Cross Linker is not always the same as the actual execution environment, nld ignores the -rpath option and RPATH set in the shared library. Please specify -Wl,-rpath-link,library-path.

Troubleshooting for execution

1. The error "Node 'N' is Offline" occurs at execution.
   The state of VE node of number N is OFFLINE. Please make it ONLINE. The example which make VE node of number 0 ONLINE state is as follows.

   # /opt/nec/ve/bin/vecmd -N 0 state set on
   ...
   Result: Success
   # /opt/nec/ve/bin/vecmd state get
   ...
   ----------------------------------------------------------------
   VE0 [03:00.0] [ ONLINE ] Last Modif:2017/11/29 10:18:00
   ----------------------------------------------------------------
   Result: Success
   



2. I want to confirm the used node at execution.
   Please execute the command /opt/nec /ve/bin/ps. The command ps outputs snapshot of executing processes by VE node. In the following example, it can be confirmed that the program named "a.out" is executing on VE node of number 2.

   /opt/nec/ve/bin/ps -a
   VE Node: 3
     PID TTY          TIME CMD
   VE Node: 1
     PID TTY          TIME CMD
   VE Node: 2
     PID TTY          TIME CMD
   50727 pts/1    00:01:36 a.out
   VE Node: 0
     PID TTY          TIME CMD
   



3. The error "./a.out: error while loading shared libraries: libncc.so.2: cannot open shared object file: No such file or directory" is output at execution.
   Please install the package "nec-nc++-shared" and "nec-nc++-shared-inst". Please follow the instructions described in the "Installation Guide".


4. The error which a dynamic link library is not found occurs at execution.
   Please set the directory which dynamic link library is put to the environment variable VE_LD_LIBRARY_PATH. Please refer to "C/C++ Compiler User's Guide".


5. The error "VE mmap failed on INTERP - TEXT: Cannot allocate memory" occurs at execution.
   The total size of text and static data of the VE program exceeds 48GB. Please modify the source files to dynamically allocate large-sized data so that the total size of the text and static data of the VE program is 48GB or less. Static data includes global variables, static variables, and initialized string literals. You can check the symbol sizes in descending order by executing the following command for reference:

   $ /opt/nec/ve/bin/nnm -C --size-sort -r ./a.out
   

   In addition, you can check the size of each section with the following command:

   $ /opt/nec/ve/bin/nreadelf -e ./a.out
   



6. I want to confirm which line of source file corresponds to an exception occurrence point.
   It can be check by traceback information. Please refer to "C/C++ Compiler User's Guide" to check process of it.


7. The exception occurrence point which output by traceback information is incorrect.
   The exception occurrence point output by traceback information can be incorrect by the advance control of HW. The advance control can be stopped to set the environment variable ADVANCEOFF=YES. An execution time may increase substantially to stop the advance control. Please take care it.

   $ export VE_ADVANCEOFF=YES
   



8. I want to confirm whether use uninitialized local variable or not.
   It may be checked by detecting an exception to compile with -minit-stack=snan and execute for double type variables. For float type variables, specify snanf instead of snan. This approach can be used only if the variable is floating-point type.


9. I want to avoid abnormal termination caused by reference of uninitialized variable.
   It may avoid by initializing the area to zero to compile with -minit-stack=zero and execute. Correction of a program is recommended to resolve a potential problem.


10. A program which uses automatic parallelization and/or OpenMP is abnormally terminated by "Unable to grow stack" or SIGSEGV at execution.
    It may occur because the amount of stack usage exceeds the limit. Please increase the limit of stack size.

    $ export OMP_STACKSIZE=2G
    



11. I want to confirm how many thread was used at execution.
    It can be confirmed to check "Max Active Threads" in PROGINF. "Max Active Threads" is output to stderr at termination when setting the environment variable VE_PROGINF=DETAIL. Please refer to "PROGINF/FTRACE user's Guide" to confirm usage of PROGINF.
    In the following example, it can be confirmed that 4 thread was used because "Max Active Threads" is 4.

    ******** Program Information ********
    (...)
    Power Throttling (sec)			:		0.000000
    Thermal Throttling (sec)		:		0.000000
    Max Active Threads			:		4
    Available CPU Cores			:		8
    Average CPU Cores Used			:		3.323850
    Memory Size Used (MB)			:		7884.000000
    Start Time (date)			:		Mon Feb 19 04:43:34 2018 JST
    End Time (date)				:		Mon Feb 19 04:44:08 2018 JST
    



12. When the threads for automatic or OpenMP parallelized program execution are created or destroyed?
    By default, the threads are created at the start of execution and destroyed at termination. The number of threads are the specified value by the environment variable OMP_NUM_THREADS or VE_OMP_NUM_THREADS. If it is not specified, the number is the same as the number of available VE cores.
    Please refer to "C/C++ Compiler User's Guide" for details.


13. When running a program that utilizes automatic or OpenMP parallelized, how is the number of threads determined at the runtime?
    The number of threads at runtime can be specified through the environment variables OMP_NUM_THREADS or VE_OMP_NUM_THREADS, the OpenMP num_threads clause, and the omp_set_num_threads() function. The priority is as follows:
    1. Value specified by num_threads clause
    2. Value specified by omp_set_num_threads() function
    3. Value specified by the environment variable VE_OMP_NUM_THREADS
    4. Value specified by the environment variable OMP_NUM_THREADS
    5. The same value as the number of available VE cores.
    The number of threads at execution is the same as the number of available VE cores if it is set a value greater than the number of available VE cores in num_threads clause, omp_set_num_threads(), VE_OMP_NUM_THREADS, or OMP_NUM_THREADS.


14. The bus errors occur when promoting vectorization.
    It may occur because vector load/store for 8 bytes elements is executed for the array aligned in 4 bytes. In the following example, the float type (aligned in 4 bytes) arrays "a" and "b" are which passed as arguments are casted to uint64_t. Therefore, it is vector load/store for 8 bytes elements.

    void func1(){
        float a[511],c[511];
    (...)
    func2(a,b);
    }
    
    void func2( void* a, void* b ){
        for(int i=255; i>0; --i){ //!!! vectorized loop
            ((uint64_t*)b)[i] = ((uint64_t*)a)[i];
        }
    }
    

    Please align them in 8 bytes as follows or specify the novector directive to the loop to stop vectorization.

    float a[511] __attribute__((aligned(8)));
    float b[511] __attribute__((aligned(8)));
    



15. I want to conform the stack size required to run the program.
    There is no way to find out the required stack size because you will not know it until you try it.

Troubleshooting for tuning

1. I want to confirm which optimization was applied to a program.
   Please refer to output diagnostics and the format list when compiling. The diagnostics list is output when the compiler option -report-diagnostics, and the format list is output when the compiler option -report-format is specified. For details, refer to "C/C++ Compiler User's Guide".


2. The performance decreases, though vectorization was promoted.
   The performance decreases by an overhead of vectorization of the few iteration loop. Please specify the novector directive to such loop to stop vectorization.


3. When automatic or OpenMP parallelized program is executed, the values displayed in the same item of PROGINF and FTRACE are different.
   The number of operations for the spin-waiting of the thread created before main program starts is added in PROGINF, but not in FTRACE.


4. When using the $omp parallel num_threads (4) and executing with the environment variable OMP_NUM_THREADS=4 or OMP_NUM_THREADS=5, the execution time with OMP_NUM_THREADS=5 is a longer than with OMP_NUM_THREADS=4. Even though there are more parallel numbers.
   When the value passed with the num_threads clause is different from the value specified with the environment variable OMP_NUM_THREADS, the execution time increases due to thread regeneration.
   Threads are automatically generated before the main program starts. The number of threads is determined by the environment variable OMP_NUM_THREADS. When the number of threads changes in the program with the function omp_set_thread_num() or num_threads clause in OpenMP, the threads generated before the main program starts is freed and the new threads are regenerated.


5. The routine name which name is "$" and number as '$1' is displayed in FTRACE output. What is it?
   It is created by compiler to do vectorization and parallelization.

Troubleshooting for installation

1. I want to check if the installation is correct.
   Please specify the --version option to check the version. If the displayed version number is the same as the installed property, it has been installed correctly. The version number is output to X.X.X in the following example.

   $ /opt/nec/ve/bin/ncc --version
   ncc (NCC) X.X.X (Build 14:10:47 Apr 23 2020)
   Copyright (C) 2018,2020 NEC Corporation.
   



2. I want to install an older version of the compiler.
   Please refer to "A.1.1 Installation of a Specific Version of the Compilers" in the SX-Aurora TSUBASA Installation Guide to install old versions of the compiler.


3. I want to use an older version of the compiler.
   Please invoke /opt/nec /ve/bin/nfort-X.X.X, ncc-X.X.X, or nc++-X.X.X (X.X.X is the version number of the compiler) at compilation.
   For detail, refer to "C/C++ Compiler User's Guide".


4. I want to start an older version of compiler by default.
   There are two ways to do it. Please choose one.
   
   1. The substance of each version of ncc /nc++/nfort commands are installed as follows. X.X.X is the version number of the compiler.

      /opt/nec/ve/ncc/X.X.X/bin/ncc
      /opt/nec/ve/ncc/X.X.X/bin/nc++
      /opt/nec/ve/nfort/X.X.X/bin/nfort
      

      Set the bin directory of the version you want to invoke by default to the command search path (environment variable PATH).
   
   
   2. Installing an older version of compiler.
   Install the package of the compiler version you want to set as default. For details, refer to "A.1.2 Change of the Compiler Versions Invoked with the Command /opt/nec/ve/bin/[nfort|ncc|nc++]" in the SX-Aurora TSUBASA Installation Guide. Please note that this method will affect all users who use /opt/nec/ve/bin/ncc (nc++, nfort).

Revision history

• Feb 29th, 2024
   Addition:
    Fortran compiler and C/C++ compiler
     Troubleshooting for execution
      13.   When running a program that utilizes automatic or
      OpenMP parallelized, how is the number of threads determined at the runtime?
   Modification:
    Fortran compiler and C/C++ compiler
     Troubleshooting for installation
      4.  I want to start an older version of compiler by default.
  
• Jun 27th, 2024
   Addition:
    Fortran compiler and C/C++ compiler
     Troubleshooting for execution
      5.   The error "VE mmap failed on INTERP - TEXT: Cannot allocate memory"
      occurs at execution.
  
• Dec 26th, 2024
   Addition:
    Fortran compiler
     Troubleshooting for installation
      5.   A message "Runtime error: vhcall_install failed." was output at execution.
  

Related Documents

• Fortran Compiler User's Guide
• C/C++ Compiler User's Guide
• PROGINF/FTRACE User's Guide
• SX-Aurora TSUBASA Installation Guide
• 
  The above related documents can be found here.