- Predstavitev
- Uporaba
- Dostop
- Programski jeziki in knjižnice
- modules
- Nadzornik računskih virov SLURM
- TotalView
- VisIt
- IOzone test
- OpenFOAM
- ppgplasma
- Inkscape
- LaTeX in SVG
- Access
- Analiza, razčlenitev in razhroščevanje
- Ansys
- BIT1
- Cluster Management Utility
- DDT
- Eclipse
- Elmer
- Goldfire Innovator
- MPI
- Mathematica
- NX shadow
- OpenMP
- Platform Managment Console
- Pošta
- R
- Subversion
- UGS NX
- Projekti in dogodki
- Prijava
- Wiki
Main menu
Nahajate se tukaj
BIT1
Program BIT1 je bil običajno serijski program s kupom grafičnih oken (xgrafix). Ker je program izjemno računsko potraten, je bil paraleliziran s pomočjo strokovnjakov v Barcelona Super Computing Centru (BSC) in s tem postal bolj uporaben, saj so serijski programi potekali tudi več mesecev. V vzporednem načinu delovanja (MPI) je grafičen način dela izključen.
Program BIT1 avtor predstavlja z: BIT1 is an electrostatic 1D3V PIC + 2D3V direct Monte Carlo code for plasma simulations. It can run in serial as well as massively parallel versions. BIT – Berkeley (the first version of the serial BIT1 was based on XPDP1 code from University of Berkeley) – Innsbruck (code has been developed by D. Tskhakaya at University of Innsbruck) – Tbilisi (D. Tskhakaya is from Tbilisi).
Namestitev BIT1
Program si pripravimo iz izvorne kode tako, da ga odpakiramo v nov imenik in pripravimo okolje za prevajanje s prevajalniki Intel in vzporednim okoljem OpenMPI:
[leon@prelog ~]$ mkdir bit1 [leon@prelog ~]$ cd bit1 [leon@prelog bit1]$ cp ../Downloads/BIT1_032010.tar . [leon@prelog bit1]$ tar xf BIT1_032010.tar [leon@prelog bit1]$ cd codeUnix [leon@prelog codeUnix]$ module load intel openmpi Loading intel/12.0 Loading openmpi/1.5.1 for compiler intel-12.0
Prevajanje
Prevajalniki Intel omogočajo napredne funkcije in boljšo optimizacijo od GCC. Za veliko število funkcij obstajajo tudi boljše zamenjave, zato pred matematično knjižnico vedno vstavimo -limf. Ker pravajalnik nima stikala -qinitauto, ki je na voljo za prevajalnike IBM (Na Mare Nostrumu v BSC) ga zaenkrat kar izklopimo, s tem da popravimo datoteko makefile tin tako zklopimo avtomatsko inicializacijo spremenljivk na nič. Ker -qinitauto=00 daje slutiti, da je v programu kakšna spremenljivka neinicializirana, bomo raje kasneje te spremenljivke poiskali.
[leon@prelog codeUnix]$ diff -u /home/leon/bit1.vanilla/codeUnix/makefile makefile --- makefile 2009-12-23 13:45:21.000000000 +0100 +++ /home/leon/bit1.fixed/codeUnix/makefile 2010-12-25 02:07:27.121474246 +0100 @@ -14,13 +14,13 @@ ## ## compiler for BSC -CC= mpicc -O2 -qinitauto=00 +CC= mpicc -O2 ## for others ##CC = cc -O2 ##CC = gcc -g -O2 ## ## -LIBS= -L/usr/X11R6/lib64 -lm -lX11 +LIBS= -L/usr/X11R6/lib64 -limf -lm -lX11 ##LIBS= -L/usr/X11R6/lib -lm -lX11 ## ## Libraries and their directories used in loading. Normal is -lm and -lX11
Program nato prevedemo z običajnim ukazom make
Poganjanje v sistemu čakalnih vrst
Za lažje pošiljanje v LSF si pripravimo skript
[leon@prelog codeUnix]$ cat bit1.lsf #!/bin/bash #BSUB -a openmpi #BSUB -n 24 #BSUB -o bit1.log #BSUB -J bit1 mpirun.lsf ./BIT1 ../tok-em4e22.inp
in ga poženemo na 24 procesorjih z ukazom
[leon@prelog codeUnix]$ bsub < bit1.lsf
Job <7> is submitted to default queue <normal>.
[leon@prelog codeUnix]$ bjobs
JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME
7 leon RUN normal prelog 12*cn52 bit1 Dec 25 15:41
12*cn55
[leon@prelog codeUnix]$ bhist -l 7|grep CPU
Sat Dec 25 15:42:05: Done successfully. The CPU time used is 885.6 seconds;
ki, se v kratkem času zaključi.
Popravki izvorne kode
Ko program konča, lahko v datoteki bit1.log opazimo, da program vedno zaključi s statusom 1 in zato LSF javlja
...
Code is based on XPDP1 code from University of California - Berkeley: Plasma Theory and Simulation Group
Final tstep= 50000
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
Job /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/openmpi_wrapper ./BIT1 ../tok-em4e22.inp
TID HOST_NAME COMMAND_LINE STATUS TERMINATION_TIME
===== ========== ================ ======================= ===================
00000 cn53 ./BIT1 ../tok-em Exit (1) 12/25/2010 13:23:15
....
zato popravimo program, da normalno konča s statusom 0 s popravkom datoteke
[leon@prelog codeUnix]$ diff -u ~leon/bit1.vanilla/codeUnix/somef.c somef.c
--- /home/leon/bit1.vanilla/codeUnix/somef.c 2009-12-23 13:45:20.000000000 +0100
+++ somef.c 2010-12-25 13:25:40.919774527 +0100
@@ -104,7 +104,7 @@
void exitrun()
{
if (fmpi) MPI_Finalize();
- exit(1);
+ exit(0);
}
/*********************************************************************
ki odpravi pritoževanje LSF tudi ob pravilnem zaključku programa.
Iskanje napak v izvorni kodi
Pri razvoju programov se napaka stalno pojavljajo. Če je le mogoče si pri iskanju pomagama z orodji, ki lahko najdejo osnovne napake. Napake pri delu s spremenljivkami nam lahko odkrijejo že pravajalniki ali orodja kot je npr. Valgrind, ki ščitijo spomin rezerviran za spremenljivke ter tako ob nekontroliranem pisanju ujamejo problematično kodo. Za prevajalnike Intel tako namesto -qinitauto napišemo iskanje neinicializiranih polj, kar deluje celo brez stikala za razhroščevanje (-g).
[leon@prelog codeUnix]$ diff -u ~leon/bit1.vanilla/codeUnix/makefile makefile --- /home/leon/bit1.vanilla/codeUnix/makefile 2009-12-23 13:45:21.000000000 +0100 +++ makefile 2010-12-25 15:55:18.208471392 +0100 @@ -14,13 +14,13 @@ ## ## compiler for BSC -CC= mpicc -O2 -qinitauto=00 +CC= mpicc -g -O2 -check-uninit -ftrapuv ## for others ##CC = cc -O2 ##CC = gcc -g -O2
Iskanje napak pričnemo s ponovnim prevajanjem in zagonom:
[leon@prelog codeUnix]$ make clean rm *.o BIT1 make -C xgrafix clean make[1]: Entering directory `/home/leon/bit1/codeUnix/xgrafix' rm xgrafix.o make[1]: Leaving directory `/home/leon/bit1/codeUnix/xgrafix' [leon@prelog codeUnix]$ make mpicc -O2 -check-uninit -ftrapuv -c -O2 fft.c mpicc -O2 -check-uninit -ftrapuv -c -O2 mcc.c ... ...
ki prikaže problematične spremenljivke v bit1.log kot:
Run-Time Check Failure: The variable 'vv' is being used without being initialized
[cn55:04806] *** Process received signal ***
[cn55:04806] Signal: Aborted (6)
[cn55:04806] Signal code: (-6)
Run-Time Check Failure: The variable 'vv' is being used without being initialized
...
S hitrim popravko oz inicializacijo spremenljivk ter poganjanju v (pseudo)ospredju
[leon@prelog codeUnix]$ grep vv *.c |grep float [leon@prelog codeUnix]$ emacs datf_disp_mpi.c datf_fv_mpi.c dmprest.c movie.c tdf.c & [leon@prelog codeUnix]$ make [leon@prelog codeUnix]$ bsub -K < bit1.lsf Job <13> is submitted to default queue <normal>. <<Waiting for dispatch ...>> <<Job is finished>>
Ko program konča lahko posmrtno (post mortem) poiščemo mesto napake tako, da poberemo zadnje "kosti" (core) in pogledamo v "pacienta" s razhroščevalnikom in pogledamo (where) na kateri vrstici se je smrt zgodila:
[leon@prelog codeUnix]$ gdb ./BIT1 core.4096 ... Program terminated with signal 6, Aborted. #0 0x0000003fc5230265 in raise () from /lib64/libc.so.6 (gdb) where #0 0x0000003fc5230265 in raise () from /lib64/libc.so.6 #1 0x0000003fc5231eb8 in abort () from /lib64/libc.so.6 #2 0x0000000000468aa0 in __intel_rtc_uninit_use () #3 0x0000000000449a01 in datf_t (datf=4096) at datf_t_mpi.c:199 #4 0x0000000000440800 in cridat_mpi (datfile=4096) at dmprest_mpi.c:449 #5 0x000000000040f4fc in main (argc=2, argv=0x7fff0aa620b8) at bit1.c:99 (gdb) quit
kar lahko potem s par poskusi ugotovimo, da je prav zaprav šlo tudi za neinicializirano spremenljivko vv1 in ne samo vv. Namesto gdb lahko uporabimo tudi Intel razghroščevalnik idb. Popravek modula datf_t_mpi.c je tako odpravljen z
[leon@prelog codeUnix]$ diff -u ~leon/bit1.vanilla/codeUnix/datf_t_mpi.c datf_t_mpi.c
--- /home/leon/bit1.vanilla/codeUnix/datf_t_mpi.c 2009-12-23 13:45:17.000000000 +0100
+++ datf_t_mpi.c 2010-12-25 17:25:06.738668233 +0100
@@ -7,7 +7,7 @@
int datf;
{
int i, j, isp;
- float vv, vv1, vv2, vv3, vv4, vv5, vv6, vv7, vv8;
+ float vv = 0.0, vv1 = 0.0, vv2, vv3, vv4, vv5, vv6, vv7, vv8;
char fn[10], fname[90];
FILE *mydmp;
in tako lahko končno poženemo program tudi brez kontrole na inicializacijo spremenljivk, saj to upočasnjuje program. Končni rezultat izpisa programa s statistiko je:
Sender: LSF System <lsfadmin@cn52>
Subject: Job 22: <bit1> Done
Job <bit1> was submitted from host <prelog> by user <leon> in cluster <hpcfs>.
Job was executed on host(s) <12*cn52>, in queue <normal>, as user <leon> in cluster <hpcfs>.
<12*cn55>
</home/leon> was used as the home directory.
</home/leon/bit1/codeUnix> was used as the working directory.
Started at Sat Dec 25 17:27:48 2010
Results reported at Sat Dec 25 17:28:48 2010
Your job looked like:
------------------------------------------------------------
# LSBATCH: User input
#!/bin/bash
#BSUB -a openmpi
#BSUB -n 24
#BSUB -o bit1.log
#BSUB -J bit1
mpirun.lsf ./BIT1 ../tok-em4e22.inp
------------------------------------------------------------
Successfully completed.
Resource usage summary:
CPU time : 965.01 sec.
Max Memory : 5670 MB
Max Swap : 11982 MB
Max Processes : 42
Max Threads : 92
The output (if any) follows:
BIT1 - Bounded Electrostatic 1 Dimensional PIC-MCC Code (Berkeley-Innsbruck-Tbilisi)
Developed in High Energy and Plasma Physics Group, University of Innsbruck
Author: D.Tskhakaya, Permanent address:
Institute of Physics, Georgian Academy of Sciences, Tbilisi
Code is based on XPDP1 code from University of California - Berkeley: Plasma Theory and Simulation Group
Final tstep= 50000
Job /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/openmpi_wrapper ./BIT1 ../tok-em4e22.inp
TID HOST_NAME COMMAND_LINE STATUS TERMINATION_TIME
===== ========== ================ ======================= ===================
00000 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00001 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00002 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00003 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00004 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00005 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00006 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00007 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00008 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00009 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00010 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00011 cn55 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00012 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00013 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00014 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00015 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00016 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00017 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00018 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00019 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00020 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00021 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00022 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
00023 cn52 ./BIT1 ../tok-em Done 12/25/2010 17:28:30
Rezultate, ki so zapisani v več datotekah *.inp.dat.<processor.rank> združimo s funkcijo zdruzi tok-em4e22.inp.dat.*
zdruzi() { echo $* | sort -t . -n -k 4 | args head -q -n -1 > izpis.txt; }
Datoteko izpis.txt kar lahko gledamo v gnuplotu z ukazom plot 'izpis.txt' using 4
Zaključek
Dodatne informacije o programu lahko dobite pri {leon.kos, nikola.jelic}@lecad.fs.uni-lj.si ali pri avtorju samem.
- Prijavi se za komentar