-
Notifications
You must be signed in to change notification settings - Fork 0
/
notes
130 lines (102 loc) · 6.14 KB
/
notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
===== original primary keys, local disk =====
2557.66user 815.13system 1:27:01elapsed 64%CPU (0avgtext+0avgdata 4870352maxresident)k
264104inputs+82136064outputs (4major+183562minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, local disk =====
3489.59user 2125.53system 2:18:35elapsed 67%CPU (0avgtext+0avgdata 15294496maxresident)k
8228192inputs+188924296outputs (52major+887259minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, local disk =====
2564.91user 1464.36system 1:08:02elapsed 98%CPU (0avgtext+0avgdata 4694688maxresident)k
620392inputs+27771520outputs (3major+264118minor)pagefaults 0swaps
===== indexes with ORDER BY postprocessing, PRAGMAs, local pre-cull, local disk =====
2666.49user 1476.42system 1:10:17elapsed 98%CPU (0avgtext+0avgdata 4669552maxresident)k
1890416inputs+27771608outputs (28major+322396minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, 4gb cache, local disk =====
2607.86user 1572.34system 1:11:45elapsed 97%CPU (0avgtext+0avgdata 4421408maxresident)k
5578160inputs+27772736outputs (27major+861648minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, default memlimit, local disk =====
MEMORY: 67062560 bytes (68085552 peak)
MEMORY: 67114784 bytes (68085552 peak)
2686.76user 2056.20system 1:21:21elapsed 97%CPU (0avgtext+0avgdata 4634512maxresident)k
3496840inputs+37648368outputs (26major+470001minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, 2gb memlimit, local disk =====
MEMORY: 1073261528 bytes (1074284520 peak)
MEMORY: 1073748152 bytes (1074284520 peak)
2698.38user 1979.49system 1:20:12elapsed 97%CPU (0avgtext+0avgdata 7898576maxresident)k
3201392inputs+37648048outputs (29major+1280397minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, on LionXG local disk =====
2335.36user 1072.20system 1:13:43elapsed 77%CPU (0avgtext+0avgdata 21485936maxresident)k
29305752inputs+70140944outputs (17major+1422631minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, on LionXG local disk, 3.5gb memlimit =====
MEMORY: 466200 bytes (1510912 peak)
MEMORY: 461952 bytes (6016520 peak)
2285.97user 1290.08system 1:00:00elapsed 99%CPU (0avgtext+0avgdata 4437040maxresident)k
56inputs+70043704outputs (1major+238113minor)pagefaults 0swaps
===== indexes with GROUP BY postprocessing, PRAGMAs, local pre-cull, on LionXG local disk, 3.5gb memlimit, dbsnp last =====
2817.80user 1981.05system 1:24:07elapsed 95%CPU (0avgtext+0avgdata 4391504maxresident)k
0inputs+45418600outputs (0major+242013minor)pagefaults 0swaps
===== full build on LionXG, 3.5gb memlimit, all on gpfs =====
MEMORY: 566232 bytes (1575448 peak)
MEMORY: 9475456 bytes (22364816 peak)
9808.98user 4032.61system 15:07:25elapsed 25%CPU (0avgtext+0avgdata 4526512maxresident)k
0inputs+24269184outputs (0major+342983minor)pagefaults 0swaps
===== full build on LionXG, 3.5gb memlimit, local disk =====
MEMORY: 566304 bytes (1575520 peak)
MEMORY: 9475528 bytes (22364888 peak)
9621.22user 2225.09system 3:43:57elapsed 88%CPU (0avgtext+0avgdata 4524320maxresident)k
15295920inputs+88378344outputs (2major+359315minor)pagefaults 0swaps
===== full build on RitchieDB, 1gb memlimit, local disk =====
MEMORY: 67179880 bytes (68158896 peak)
MEMORY: 67443264 bytes (71458976 peak)
8680.24user 2299.44system 3:26:03elapsed 88%CPU (0avgtext+0avgdata 4788688maxresident)k
1808inputs+101156576outputs (0major+333351minor)pagefaults 0swaps
===== single no-ECR build on ACCRE, cache-only, 3.5gb memlimit, all on gpfs =====
MEMORY: 547952 bytes (1578096 peak)
MEMORY: 9470984 bytes (22350408 peak)
4640.29user 3065.78system 2:27:11elapsed 87%CPU (0avgtext+0avgdata 4522192maxresident)k
14728inputs+30787680outputs (0major+320317minor)pagefaults 0swaps
===== single no-ECR build on ACCRE, with download, 3.5gb memlimit, all on gpfs =====
MEMORY: 547952 bytes (1575416 peak)
MEMORY: 9470984 bytes (22353392 peak)
4807.82user 3012.10system 3:14:07elapsed 67%CPU (0avgtext+0avgdata 4523904maxresident)k
144inputs+35803480outputs (0major+312840minor)pagefaults 0swaps
===== separate no-ECR build on ACCRE, cache-only, 3.5gb memlimit, all on gpfs =====
resources_used.cput=03:08:06
resources_used.mem=1744852kb
resources_used.vmem=2245956kb
resources_used.walltime=03:34:49
===== original primary keys, separate no-ECR build on ACCRE =====
2 days, +over 3 hours for schema rollup
===== single no-ECR build on ACCRE =====
MEMORY: 67161896 bytes (68158952 peak)
MEMORY: 67439064 bytes (71327176 peak)
4591.11user 2882.00system 2:26:32elapsed 84%CPU (0avgtext+0avgdata 4787200maxresident)k
272inputs+45065392outputs (0major+316971minor)pagefaults 0swaps
===== ldprofiles on LionXG, 2 pops, 8 values, local disk =====
17018.12user 176.34system 5:03:30elapsed 94%CPU (0avgtext+0avgdata 6838240maxresident)k
160inputs+120079888outputs (2major+6882985minor)pagefaults 0swaps
===== ldprofiles on ACCRE, 2 pops, 8 values, gpfs =====
25296.87user 377.90system 8:43:51elapsed 81%CPU (0avgtext+0avgdata 6838096maxresident)k
240inputs+88976outputs (2major+6909967minor)pagefaults 0swaps
===== finalize on LionXG, local disk =====
288.07user 55.77system 8:26.16elapsed 67%CPU (0avgtext+0avgdata 309328maxresident)k
520inputs+50878416outputs (0major+20199minor)pagefaults 0swaps
===== finalize on LionXG, gpfs =====
285.12user 82.22system 25:14.84elapsed 24%CPU (0avgtext+0avgdata 309360maxresident)k
0inputs+12506232outputs (0major+20202minor)pagefaults 0swaps
===== finalize on ACCRE, gpfs =====
415.53user 79.68system 43:51.48elapsed 18%CPU (0avgtext+0avgdata 308704maxresident)k
0inputs+11454392outputs (0major+20985minor)pagefaults 0swaps
===== as of rc2 =====
CaptainHammer (Intel Xeon E5606 2.13 GHz; local disk)
full data build: 5h, ~1.2gb RAM
2pop 6val LD: 8h, ~1.7gb RAM
finalize: 25m, ~100kb RAM
LionXG (Intel Xeon E5-2665 2.40 GHz; local disk)
full data build: 3h10m, ~1.2gb RAM
2pop 6val LD: 5h15m, ~1.7gb RAM
finalize: 8m, ~100kb RAM
ACCRE (AMD Opteron 2.4 GHz; GPFS)
non-ECR build: 2h10m, ~1.2gb RAM
2pop 4val LD: 8h15m, ~1.7gb RAM
2pop 6val LD: 9h15m, ~1.7gb RAM (Intel Nehalem 2.3 GHz)
finalize: 20m, ~100kb RAM