There are 6 columns entries in a file, where each column specifies values for(days, hours, Temperature, Relative Humidity, wind speed, global horizontal solar radiation) respectively. How we can store these values in array or vector of objects ? please help
1 1 13.7 58 2.7 0
1 2 13.5 64 1.4 0
1 3 13 70 0 0
1 4 12.2 75 0.5 0
1 5 11.4 80 1 0
1 6 10.6 85 1.5 0
1 7 11.1 80 1 0
1 8 11.5 78 0.5 13
1 9 12 76 0 150
1 10 15.1 76 1 355
1 11 18.3 73 2.1 532
1 12 21.4 70 3.1 652
1 13 21.9 62 2.9 706
1 14 22.5 56 2.8 686
1 15 23 49 2.6 593
1 16 22.6 50 2.4 434
1 17 22.2 52 2.3 234
1 18 21.8 53 2.1 45
1 19 19.9 57 1.4 0
1 20 17.9 60 0.7 0
1 21 16 63 0 0
1 22 15.7 60 1.2 0
1 23 15.5 56 2.4 0
1 24 15.2 53 3.6 0
2 1 14.1 58 2.4 0
2 2 13.1 63 1.2 0
2 3 12 69 0 0
2 4 11.1 74 0 0
2 5 10.1 79 0 0
2 6 9.2 84 0 0
2 7 9.9 79 0.3 0
2 8 10.7 75 0.7 13
2 9 11.4 71 1 150
2 10 13.5 60 1.3 358
2 11 15.6 51 1.7 539
2 12 17.7 43 2.1 664
2 13 19.8 37 2.4 718
2 14 21.9 31 2.7 697
2 15 24 26 3.1 603
2 16 23.7 27 2.8 443
2 17 23.3 28 2.4 240
2 18 23 29 2.1 47
2 19 21.1 35 1.9 0
2 20 19.1 42 1.7 0
2 21 17.2 50 1.5 0
2 22 16.1 53 1.5 0
2 23 15.1 57 1.5 0
2 24 14 61 1.5 0
std::getline to extract each line, and std::stringstream to extract each value from this line:
#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <vector>
struct Data {
unsigned int days, hours;
float humidity, windSpeed, radiation;
};
int main()
{
std::vector<Data> data;
std::ifstream file("yourfile.txt");
for (std::string line; std::getline(file, line); )
{
Data d;
std::stringstream ss(line);
ss >> d.days >> d.hours >> d.humidity >> d.windSpeed >> d.radiation;
data.push_back(d);
}
}
Trying to skip i rows of a data file (then process j rows). Plenty of answers on length of csv file. No success with count or .line_num.
How is current line number accessed in csv.reader?
import csv
def csv_reader(df):
i = 90
with open(df, 'r') as csvfile:
for line in range(0, i):
next(csvfile)
for line in csv.reader(csvfile, delimiter=' ', skipinitialspace = True):
print(csv.reader.line_num) # Invalid line
Sample data:
Last Quarter
Visit Astronomy
Daily Weather History & Observations
2018 Temp. (°C) Dew Point (°C) Humidity (%) Sea Level Press. (hPa) Visibility (km) Wind (km/h) Precip. (mm) Events
Mar high avg low high avg low high avg low high avg low high avg low high avg high sum
1 27 19 12 13 9 4 82 49 21 1016 1012 1007 10 10 10 24 11 - 0.00
2 25 20 14 14 12 9 82 61 32 1017 1014 1010 10 10 10 21 10 - 0.00
3 31 22 14 15 13 7 94 59 13 1014 1011 1007 10 10 10 27 8 - 0.00
4 30 21 13 15 13 6 82 59 13 1016 1012 1009 10 10 10 34 11 - 0.00
5 24 19 15 16 13 11 82 71 46 1022 1016 1013 10 10 10 35 13 - 0.00 Rain
6 20 14 9 12 9 6 82 60 31 1028 1024 1021 10 10 10 32 19 47 0.00
7 23 16 9 13 10 7 100 71 29 1029 1027 1024 10 10 5 29 11 37 0.00 Rain
Other random data follows
You can access line number using enumerate.
for index, line in enumerate(csv.reader(csvfile, delimiter=' ', skipinitialspace = True)):
print('Index %s' % str(index + 1)) # enumerate starts with 0
First please consider the following list of WC variable...
1101 1201 1301 1401 1501 1601
1102 1202 1302 1402 1502 1602
1103 1203 1303 1403 1503 1603
1104 1204 1304 1404 1504 1604
1105 1205 1305 1405 1505 1605
1106 1206 1306 1406 1506 1606
1107 1207 1307 1407 1507 1607
1108 1208 1308 1408 1508 1608
1109 1209 1309 1409 1509 1609
1110 1210 1310 1410 1510 1610
1111 1211 1311 1411 1511 1611
1112 1212 1312 1412 1512 1612
1113 1213 1313 1413 1513 1613
1114 1214 1314 1414 1514 1614
1115 1215 1315 1415 1515 1615
1116 1216 1316 1416 1516 1616
1117 1217 1317 1417 1517 1617
1118 1218 1318 1418 1518 1618
1119
1120
Now understand what I am trying to achieve.
I have a very long dataset where in i have two variables are shown in the below table...
WC ASN
1101 0
1101 1
1101 2
1101 3
1101 4
1101 20
1101 21
1101 22
1101 23
1101 24
1101 25
1101 26
1101 27
1101 28
1101 45
1101 46
1101 47
1101 48
1201 4
1201 5
1201 6
1201 7
1201 8
1201 16
1201 17
1201 18
1201 19
1201 20
1201 28
1201 29
1201 30
1201 31
1201 32
1201 41
1201 42
1201 43
1201 44
1202 4
1202 5
1202 6
1202 7
1202 8
1202 16
1202 17
1202 18
1202 19
1202 20
1202 29
1202 30
1202 31
1202 32
1202 40
1202 41
1202 42
1202 43
1202 44
What I want to do is to add two more columns Group and SubGroup such that I get the final table as shown below:
WC ASN Group SubGroup
1101 0 1 1
1101 1 1 1
1101 2 1 1
1101 3 1 1
1101 4 1 1
1101 20 1 1
1101 21 1 1
1101 22 1 1
1101 23 1 1
1101 24 1 1
1101 25 1 1
1101 26 1 1
1101 27 1 1
1101 28 1 1
1101 45 1 1
1101 46 1 1
1101 47 1 1
1101 48 1 1
1201 4 1 2
1201 5 1 2
1201 6 1 2
1201 7 1 2
1201 8 1 2
1201 16 1 2
1201 17 1 2
1201 18 1 2
1201 19 1 2
1201 20 1 2
1201 28 1 2
1201 29 1 2
1201 30 1 2
1201 31 1 2
1201 32 1 2
1201 41 1 2
1201 42 1 2
1201 43 1 2
1201 44 1 2
1301 8 1 3
1301 9 1 3
1301 10 1 3
1301 11 1 3
1301 12 1 3
1301 13 1 3
1301 14 1 3
1301 15 1 3
1301 16 1 3
1301 32 1 3
1301 33 1 3
1301 34 1 3
1301 35 1 3
1301 36 1 3
1301 37 1 3
1301 38 1 3
1301 39 1 3
1301 40 1 3
1401 8 1 4
1401 9 1 4
1401 10 1 4
1401 11 1 4
1401 12 1 4
1401 13 1 4
1401 14 1 4
1401 15 1 4
1401 16 1 4
1401 33 1 4
1401 34 1 4
1401 35 1 4
1401 36 1 4
1401 37 1 4
1401 38 1 4
1401 39 1 4
1401 40 1 4
1501 4 1 5
1501 5 1 5
1501 6 1 5
1501 7 1 5
1501 8 1 5
1501 16 1 5
1501 17 1 5
1501 18 1 5
1501 19 1 5
1501 20 1 5
1501 29 1 5
1501 30 1 5
1501 31 1 5
1501 32 1 5
1501 40 1 5
1501 41 1 5
1501 42 1 5
1501 43 1 5
1501 44 1 5
1601 0 1 6
1601 1 1 6
1601 2 1 6
1601 3 1 6
1601 4 1 6
1601 20 1 6
1601 21 1 6
1601 22 1 6
1601 23 1 6
1601 24 1 6
1601 25 1 6
1601 26 1 6
1601 27 1 6
1601 28 1 6
1601 44 1 6
1601 45 1 6
1601 46 1 6
1601 47 1 6
1601 48 1 6
I was trying something like this...
select;
when (WC = 1101) group = 1 subgroup=1;
when (WC = 1201) group = 1 subgroup=2;
when (WC = 1301) group = 1 subgroup=3;
when (WC = 1401) group = 1 subgroup=4;
when (WC = 1501) group = 1 subgroup=5;
when (WC = 1601) group = 1 subgroup=6;
when (WC = 1102) group = 2 subgroup=1;
when (WC = 1202) group = 2 subgroup=2;
when (WC = 1302) group = 2 subgroup=3;
when (WC = 1402) group = 2 subgroup=4;
.
.
.
when (WC = 1617) group = 18 subgroup=5;
when (WC = 1618) group = 18 subgroup=6;
when (WC = 1119) group = 1 subgroup=1;
otherwise group = 20 subgroup=1;
end;
This is seriously long, tedious and confusing. I am sure there are better and short ways of doing it.
Please help.
As Joe and Keith suggested, I am adding this explanation.
WC in the table as the appear..
WC1 WC2 WC3 WC4 WC5 WC6 Group
1101 1201 1301 1401 1501 1601 1
1102 1202 1302 1402 1502 1602 2
1103 1203 1303 1403 1503 1603 3
.
.
.
WC ASN Group SubGroup
1101 0 1 1 (because last two digits are 01 and 2nd degit is 1)
.
.
1206 1 6 2
.
.
1201 2 1 2
.
.
1213 3 13 2
.
.
1610 4 10 6
Seems like a case for substr to me.
group = substr(wc,1,1);
subgroup = substr(wc,2,1);
Or whatever your rule is. If your rule is more complicated than that, explain the rule.
Which is faster in GLSL:
pow(x, 3.0f);
or
x*x*x;
?
Does exponentiation performance depend on hardware vendor or exponent value?
I wrote a small benchmark, because I was interested in the results.
In my personal case, I was most interested in exponent = 5.
Benchmark code (running in Rem's Studio / LWJGL):
package me.anno.utils.bench
import me.anno.gpu.GFX
import me.anno.gpu.GFX.flat01
import me.anno.gpu.RenderState
import me.anno.gpu.RenderState.useFrame
import me.anno.gpu.framebuffer.Frame
import me.anno.gpu.framebuffer.Framebuffer
import me.anno.gpu.hidden.HiddenOpenGLContext
import me.anno.gpu.shader.Renderer
import me.anno.gpu.shader.Shader
import me.anno.utils.types.Floats.f2
import org.lwjgl.opengl.GL11.*
import java.nio.ByteBuffer
import kotlin.math.roundToInt
fun main() {
fun createShader(code: String) = Shader(
"", null, "" +
"attribute vec2 attr0;\n" +
"void main(){\n" +
" gl_Position = vec4(attr0*2.0-1.0, 0.0, 1.0);\n" +
" uv = attr0;\n" +
"}", "varying vec2 uv;\n", "" +
"void main(){" +
code +
"}"
)
fun repeat(code: String, times: Int): String {
return Array(times) { code }.joinToString("\n")
}
val size = 512
val warmup = 50
val benchmark = 1000
HiddenOpenGLContext.setSize(size, size)
HiddenOpenGLContext.createOpenGL()
val buffer = Framebuffer("", size, size, 1, 1, true, Framebuffer.DepthBufferType.NONE)
println("Power,Multiplications,GFlops-multiplication,GFlops-floats,GFlops-ints,GFlops-power,Speedup")
useFrame(buffer, Renderer.colorRenderer) {
RenderState.blendMode.use(me.anno.gpu.blending.BlendMode.ADD) {
for (power in 2 until 100) {
// to reduce the overhead of other stuff
val repeats = 100
val init = "float x1 = dot(uv, vec2(1.0)),x2,x4,x8,x16,x32,x64;\n"
val end = "gl_FragColor = vec4(x1,x1,x1,x1);\n"
val manualCode = StringBuilder()
for (bit in 1 until 32) {
val p = 1.shl(bit)
val h = 1.shl(bit - 1)
if (power == p) {
manualCode.append("x1=x$h*x$h;")
break
} else if (power > p) {
manualCode.append("x$p=x$h*x$h;")
} else break
}
if (power.and(power - 1) != 0) {
// not a power of two, so the result isn't finished yet
manualCode.append("x1=")
var first = true
for (bit in 0 until 32) {
val p = 1.shl(bit)
if (power.and(p) != 0) {
if (!first) {
manualCode.append('*')
} else first = false
manualCode.append("x$p")
}
}
manualCode.append(";\n")
}
val multiplications = manualCode.count { it == '*' }
// println("$power: $manualCode")
val shaders = listOf(
// manually optimized
createShader(init + repeat(manualCode.toString(), repeats) + end),
// can be optimized
createShader(init + repeat("x1=pow(x1,$power.0);", repeats) + end),
// can be optimized, int as power
createShader(init + repeat("x1=pow(x1,$power);", repeats) + end),
// slightly different, so it can't be optimized
createShader(init + repeat("x1=pow(x1,${power}.01);", repeats) + end),
)
for (shader in shaders) {
shader.use()
}
val pixels = ByteBuffer.allocateDirect(4)
Frame.bind()
glClearColor(0f, 0f, 0f, 1f)
glClear(GL_COLOR_BUFFER_BIT or GL_DEPTH_BUFFER_BIT)
for (i in 0 until warmup) {
for (shader in shaders) {
shader.use()
flat01.draw(shader)
}
}
val flops = DoubleArray(shaders.size)
val avg = 10 // for more stability between runs
for (j in 0 until avg) {
for (index in shaders.indices) {
val shader = shaders[index]
GFX.check()
val t0 = System.nanoTime()
for (i in 0 until benchmark) {
shader.use()
flat01.draw(shader)
}
// synchronize
glReadPixels(0, 0, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, pixels)
GFX.check()
val t1 = System.nanoTime()
// the first one may be an outlier
if (j > 0) flops[index] += multiplications * repeats.toDouble() * benchmark.toDouble() * size * size / (t1 - t0)
GFX.check()
}
}
for (i in flops.indices) {
flops[i] /= (avg - 1.0)
}
println(
"" +
"$power,$multiplications," +
"${flops[0].roundToInt()}," +
"${flops[1].roundToInt()}," +
"${flops[2].roundToInt()}," +
"${flops[3].roundToInt()}," +
(flops[0] / flops[3]).f2()
)
}
}
}
}
The sampler function is run 9x 512² pixels * 1000 times, and evaluates the function 100 times each.
I run this code on my RX 580, 8GB from Gigabyte, and collected the following results:
Power
#Mult
GFlops*
GFlopsFp
GFlopsInt
GFlopsPow
Speedup
2
1
1246
1429
1447
324
3.84
3
2
2663
2692
2708
651
4.09
4
2
2682
2679
2698
650
4.12
5
3
2766
972
974
973
2.84
6
3
2785
978
974
976
2.85
7
4
2830
1295
1303
1299
2.18
8
3
2783
2792
2809
960
2.90
9
4
2836
1298
1301
1302
2.18
10
4
2833
1291
1302
1298
2.18
11
5
2858
1623
1629
1623
1.76
12
4
2824
1302
1295
1303
2.17
13
5
2866
1628
1624
1626
1.76
14
5
2869
1614
1623
1611
1.78
15
6
2886
1945
1943
1953
1.48
16
4
2821
1305
1300
1305
2.16
17
5
2868
1615
1625
1619
1.77
18
5
2858
1620
1625
1624
1.76
19
6
2890
1949
1946
1949
1.48
20
5
2871
1618
1627
1625
1.77
21
6
2879
1945
1947
1943
1.48
22
6
2886
1944
1949
1952
1.48
23
7
2901
2271
2269
2268
1.28
24
5
2872
1621
1628
1624
1.77
25
6
2886
1942
1943
1942
1.49
26
6
2880
1949
1949
1953
1.47
27
7
2891
2273
2263
2266
1.28
28
6
2883
1949
1946
1953
1.48
29
7
2910
2279
2281
2279
1.28
30
7
2899
2272
2276
2277
1.27
31
8
2906
2598
2595
2596
1.12
32
5
2872
1621
1625
1622
1.77
33
6
2901
1953
1942
1949
1.49
34
6
2895
1948
1939
1944
1.49
35
7
2895
2274
2266
2268
1.28
36
6
2881
1937
1944
1948
1.48
37
7
2894
2277
2270
2280
1.27
38
7
2902
2275
2264
2273
1.28
39
8
2910
2602
2594
2603
1.12
40
6
2877
1945
1947
1945
1.48
41
7
2892
2276
2277
2282
1.27
42
7
2887
2271
2272
2273
1.27
43
8
2912
2599
2606
2599
1.12
44
7
2910
2278
2284
2276
1.28
45
8
2920
2597
2601
2600
1.12
46
8
2920
2600
2601
2590
1.13
47
9
2925
2921
2926
2927
1.00
48
6
2885
1935
1955
1956
1.47
49
7
2901
2271
2279
2288
1.27
50
7
2904
2281
2276
2278
1.27
51
8
2919
2608
2594
2607
1.12
52
7
2902
2282
2270
2273
1.28
53
8
2903
2598
2602
2598
1.12
54
8
2918
2602
2602
2604
1.12
55
9
2932
2927
2924
2936
1.00
56
7
2907
2284
2282
2281
1.27
57
8
2920
2606
2604
2610
1.12
58
8
2913
2593
2597
2587
1.13
59
9
2925
2923
2924
2920
1.00
60
8
2930
2614
2606
2613
1.12
61
9
2932
2946
2946
2947
1.00
62
9
2926
2935
2937
2947
0.99
63
10
2958
3258
3192
3266
0.91
64
6
2902
1957
1956
1959
1.48
65
7
2903
2274
2267
2273
1.28
66
7
2909
2277
2276
2286
1.27
67
8
2908
2602
2606
2599
1.12
68
7
2894
2272
2279
2276
1.27
69
8
2923
2597
2606
2606
1.12
70
8
2910
2596
2599
2600
1.12
71
9
2926
2921
2927
2924
1.00
72
7
2909
2283
2273
2273
1.28
73
8
2909
2602
2602
2599
1.12
74
8
2914
2602
2602
2603
1.12
75
9
2924
2925
2927
2933
1.00
76
8
2904
2608
2602
2601
1.12
77
9
2911
2919
2917
2909
1.00
78
9
2927
2921
2917
2935
1.00
79
10
2929
3241
3246
3246
0.90
80
7
2903
2273
2276
2275
1.28
81
8
2916
2596
2592
2589
1.13
82
8
2913
2600
2597
2598
1.12
83
9
2925
2931
2926
2913
1.00
84
8
2917
2598
2606
2597
1.12
85
9
2920
2916
2918
2927
1.00
86
9
2942
2922
2944
2936
1.00
87
10
2961
3254
3259
3268
0.91
88
8
2934
2607
2608
2612
1.12
89
9
2918
2939
2931
2916
1.00
90
9
2927
2928
2920
2924
1.00
91
10
2940
3253
3252
3246
0.91
92
9
2924
2933
2926
2928
1.00
93
10
2940
3259
3237
3251
0.90
94
10
2928
3247
3247
3264
0.90
95
11
2933
3599
3593
3594
0.82
96
7
2883
2282
2268
2269
1.27
97
8
2911
2602
2595
2600
1.12
98
8
2896
2588
2591
2587
1.12
99
9
2924
2939
2936
2938
1.00
As you can see, a power() call takes exactly as long as 9 multiplication instructions. Therefore every manual rewriting of a power with less than 9 multiplications is faster.
Only the cases 2, 3, 4, and 8 are optimized by my driver. The optimization is independent of whether you use the .0 suffix for the exponent.
In the case of exponent = 2, my implementation seems to have lower performance than the driver. I am not sure, why.
The speedup is the manual implementation compared to pow(x,exponent+0.01), which cannot be optimized by the compiler.
Because the multiplications and the speedup align so perfectly, I created a graph to show the relationship. This relationship kind of shows that my benchmark is trustworthy :).
Operating System: Windows 10 Personal
GPU: RX 580 8GB from Gigabyte
Processor: Ryzen 5 2600
Memory: 16 GB DDR4 3200
GPU Driver: 21.6.1 from 17th June 2021
LWJGL: Version 3.2.3 build 13
While this can definitely be hardware/vendor/compiler dependent, advanced mathematical functions like pow() tend to be considerably more expensive than basic operations.
The best approach is of course to try both, and benchmark. But if there is a simple replacement for an advanced mathematical functions, I don't think you can go very wrong by using it.
If you write pow(x, 3.0), the best you can probably hope for is that the compiler will recognize the special case, and expand it. But why take the risk, if the replacement is just as short and easy to read? C/C++ compilers don't always replace pow(x, 2.0) by a simple multiplication, so I wouldn't necessarily count on all GLSL compilers to do that.
I will try to explain what I need help with.
The numbers in below series that I want to check by the regex, are "2 904", "3 231", "2 653", "2 653", "2 353" and so on. My goal is to only get a match if one of these numbers will be in format "123" (3-digits, between 100-999)
No match:
sö 31 1 2 904 2 3 231 3 2 653 32 4 2 653 5 2 353 6 2 353 7 2 353 8 2 904 9 3 002 10 3 143 33 11 2 615 12 2 353 13 2 353 14 2 353 15 2 353 16 2 653 17 2 353 34 18 2 157 19 1 699 20 1 699
Match:
sö 31 1 2 904 2 3 231 3 653 32 4 2 653 5 2 353 6 2 353 7 2 353 8 2 904 9 3 002 10 3 143 33 11 2 615 12 2 353 13 2 353 14 2 353 15 2 353 16 2 653 17 2 353 34 18 2 157 19 1 699 20 1 699
sö 31 1 2 904 2 3 231 3 2 653 32 4 2 653 5 2 353 6 2 353 7 2 353 8 2 904 9 3 002 10 3 143 33 11 2 615 12 2 353 13 2 353 14 953 15 2 353 16 2 653 17 2 353 34 18 2 157 19 1 699 20 1 699
As you can see from my examples, number "2 653" changed to "653" just after number "3"
And number " 2 353" changed to "953" after number "14".
Numbers between, i.e 1-20, are static numbers and will never change
Possible?
I will try it then at http://rubular.com/