def my_method(options = {})
# ...
end
# => Syntax error in ./src/auto_harvest.cr:17: for empty hashes use '{} of KeyType => ValueType'
While this is valid Ruby it seems not to be in Crystal, my suspicion is that it is because of typing. How do I tell compiler I want to default to an empty hash?
Use a default argument (like in Ruby):
def my_method(x = 1, y = 2)
x + y
end
my_method x: 10, y: 20 #=> 30
my_method x: 10 #=> 12
my_method y: 20 #=> 21
Usage of hashes for default/named arguments is totally discouraged in Crystal
(edited to include the sample instead of linking to the docs)
It seems the error has all the information I needed, I need to specify the type for the key and values of the Hash.
def my_method(options = {} of Symbol => String)
# ...
end
It is quite clearly in the docs too.
Related
I have some data which includes sizes, much like the model below.
class Product(models.Model):
width = models.CharField()
height = models.CharField()
length = models.CharField()
Through annotation we have a field called at_size which produces data like:
[None, None, None]
['200', '000', '210']
['180', None, None]
This was accomplished like so (thanks to: )https://stackoverflow.com/a/70266320/5731101:
class Array(Func):
template = '%(function)s[%(expressions)s]'
function = 'ARRAY'
out_format = ArrayField(CharField(max_length=200))
annotated_qs = Product.objects.all().annotate(
at_size=Array(F('width'), F('height'), F('length'),
output_field=out_format)
)
I'm trying to get this to convert into:
''
'200 x 000 x 210'
'180'
In code, this could a bit like ' x '.join([i for i in data if i]). But as I need to accomplish this with database functions it's a bit more challenging.
I've been playing with StringAgg, but I keep getting:
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
It looks like I need to make sure the None values are excluded from the initial Array-func to begin with. But I'm not sure where to get started here.
How can I accomplish this?
Turns out the problem was two-fold.
Cleaning out the Null values could be done through using array_remove
glueing the strings with a delimiter through StringAgg only works if the input are strings. But since we use an array, that wasn't the way to go. Instead use array_to_string
The final result looks like:
class Array(Func):
# https://www.postgresql.org/docs/9.6/functions-array.html
template = '%(function)s[%(expressions)s]'
function = 'ARRAY'
class ArrayRemove(Func):
# https://www.postgresql.org/docs/9.6/functions-array.html
function = 'array_remove'
class ArrayToString(Func):
# https://stackoverflow.com/a/57873772/5731101
function = 'array_to_string'
out_format = ArrayField(CharField(max_length=200))
annotated_qs = annotated_qs.annotate(
at_size=ArrayToString(
ArrayRemove(
Array(F('width'), F('height'), F('length'), output_field=out_format),
None, # Remove None values from the Array with ArrayRemove
),
Value(" x "), # Delimiter.
Value(''), # If there are null-values, replace with... (fallback)
output_field=CharField(max_length=200),
)
)
this produces the desired format:
for product in annotated_qs:
print(product.at_size)
180 x 000 x 200
180 x 026 x 200
180 x 7 x 200
180 x 000 x 200
200 x 000 x 220
180 x 000 x 200
175 x 230 x 033
160 x 000 x 200
60 x 220
Product.objects.annotate(display_format=Concat(F('width'), '×', F('height'), '×', F('length')))
Should do the trick no?
No need to over complicate this, let's keep it nice and simple and use the database to concatenate the 3 strings and separate them with the multiplication symbol (obviously replace if you prefer another character).
Take a look at the docs over here:
https://docs.djangoproject.com/en/4.1/ref/models/database-functions/#concat
What does the ** prefix do in this method call using Crystal-lang? This is from the shrine file package. Can you explain how I would use a double splat?
class FileImport::AssetUploader < Shrine
def generate_location(io : IO | UploadedFile, metadata, context, **options) HERE
name = super(io, metadata, **options)
File.join("imports", context[:model].id.to_s, name)
end
end
FileImport::AssetUploader.upload(file, "store", context: { model: YOUR_ORM_MODEL } })
According to the official docs:
A double splat (**) captures named arguments that were not matched by
other parameters. The type of the parameter is a NamedTuple.
def foo(x, **other)
# Return the captured named arguments as a NamedTuple
return other
end
foo 1, y: 2, z: 3 # => {y: 2, z: 3}
foo y: 2, x: 1, z: 3 # => {y: 2, z: 3}
The usefulness of the double splat is that it captures all named arguments. For example, you may create a function that handles any number of keyword arguments.
def print_any_tuple_with_any_keys(**named_tuple)
named_tuple.each { |k, v| puts "Options #{k}: #{v}" }
end
print_any_tuple_with_any_keys(api: "localhost")
print_any_tuple_with_any_keys(fruit: "banana", color: "yellow")
print_any_tuple_with_any_keys(hash: "123", power: "2", cypher: "AES")
This will output:
Options api: localhost
Options fruit: banana
Options color: yellow
Options hash: 123
Options power: 2
Options cypher: AES
In the code you provided, all the other named arguments passed to generate_location that do not match io, metadata, or context will be passed down to the super function that is calling the parent class, in this case, a Shrine class.
The use for Shrine specifically, is that they provide a generic upload function for different storage engines, any extra arguments may or may not be used down the call tree, and in the case of AWS S3 storage, there may be a metadata argument that adds metadata to the file.
I'm trying to do a very simple program to find matches with ismatch() function in Julia. Suppose my pattern is
e_pat = r".+#.+"
Then I make a list called input with some random elements:
input= ["pipo#gmail.com", 23, "trapo", "holi#gmail.com"]
Now I want to identify how many matches exist and then print them using e_pat as reference:
for i in input
println(ismatch(e_pat, i)) && println(i)
end
With that code I just get "true" and the error displayed below:
true
TypeError: non-boolean (Void) used in boolean context
Stacktrace:
[1] macro expansion at ./In[27]:4 [inlined]
[2] anonymous at ./<missing>:?
[3] include_string(::String, ::String) at ./loading.jl:522
What can I do in order to get the following?
"pipo#gmail.com"
"holi#gmail.com"
I read ismatch() documentation but found nothing useful.
Any help will be much appreciated
The problem is that while this expression returns true:
julia> #show ismatch(e_pat, "pipo#gmail.com");
ismatch(e_pat,"pipo#gmail.com") = true
Using println, just prints true but returns nothing:
julia> #show println(ismatch(e_pat, "pipo#gmail.com"));
true
println(ismatch(e_pat,"pipo#gmail.com")) = nothing
Which is of type Void:
julia> typeof(nothing)
Void
And the error is telling you that you cant use an object of type Void in a boolean context (nothing) is just an instance of Void treated like a singleton in Julia:
julia> nothing && true
ERROR: TypeError: non-boolean (Void) used in boolean context
After fixing that also notice that this is also another error:
julia> #show ismatch(e_pat, 42);
ERROR: MethodError: no method matching ismatch(::Regex, ::Int32)
Closest candidates are:
ismatch(::Regex, ::SubString{T<:AbstractString}) at regex.jl:151
ismatch(::Regex, ::SubString{T<:AbstractString}, ::Integer) at regex.jl:151
ismatch(::Regex, ::AbstractString) at regex.jl:145
...
This is telling you that ismatch has no such method, you can't use it with a combination of arguments of types: (Regex, Int).
You could do something like this instead to make sure all the objects are Strings:
julia> input = string.(["pipo#gmail.com", 23, "trapo", "holi#gmail.com"])
4-element Array{String,1}:
"pipo#gmail.com"
"23"
"trapo"
"holi#gmail.com"
Finally, you could use the macro #show (which prints an expression and its result and finally returns the result) instead of the println function (which prints the result and returns nothing, to debug whats going on:
julia> for i in input
#show(ismatch(e_pat, i)) && println(i)
end
ismatch(e_pat,i) = true
pipo#gmail.com
ismatch(e_pat,i) = false
ismatch(e_pat,i) = false
ismatch(e_pat,i) = true
holi#gmail.com
So in order to print your expected result just remove the left hand side println:
julia> for i in input
ismatch(e_pat, i) && println(i)
end
pipo#gmail.com
holi#gmail.com
If you want to store them instead of printing them you could us an array comprehension instead:
julia> result = [str for str in input if ismatch(e_pat, str)]
2-element Array{String,1}:
"pipo#gmail.com"
"holi#gmail.com"
Or an indexing expression like this one:
julia> ismatch.(e_pat, input)
4-element BitArray{1}:
true
false
false
true
julia> result = input[ismatch.(e_pat, input)]
2-element Array{String,1}:
"pipo#gmail.com"
"holi#gmail.com"
That way you could print them later without having to repeat the computation:
julia> println.(result)
pipo#gmail.com
holi#gmail.com
I am finding some weird behavior when using index.
#Defined in the class's initialize
#my_list = [] of Type
index = #my_list.index { |i| i.value == 2 } # => 0
#my_list[0] # => 2
#my_list[index] # => error
I get the error:
no overload matches 'Array(Type)#[]' with type (Int32 | Nil)
Not sure why index does not work, as index = 0.
EDIT:
More information. If I do this:
if index == nil
#Do something
#error => undefined method '>=' for Nil (compile-time type is (Int32 | Nil))
elsif index >= 0
#Do something else
end
Which I understand. It could be nil, but since I'm already checking to see if it is nil there shouldn't be a problem here. I'm thinking the previous code snippet is running into the same issue.
The problem is that Array#index is nilable; it may not find anything and return nil, hence it returns an Int32|Nil union.
The compiler eventually fails because Array#[] expects an Int32 argument, but we pass it an Int32|Nil. You must take care of this case (to avoid later bugs) by checking if the return value is truthy for example.
As stated by #julien-portalier :
The problem is that Array#index is nilable; it may not find anything in the array and return Nil, hence it returns an (Int32 | Nil) union.
You can use Object#not_nil! to get ride of the Nil type :
#my_list = [2, 4, 6, 8]
index = #my_list.index { |i| i == 6 }.not_nil! # => 2
# compile type of 'index' is Int32
#my_list[0] # => 2
#my_list[index] # => 6
It will make sure the type returned by Array#index is not Nil, if it is, an exception will be raised (see Nil#not_nil!)
And if you need to handle an index error without using exceptions, you can simply check if Array#index failed :
#my_list = [2, 4, 6, 8]
index = #my_list.index { |i| i == 6 } # => 2
# compile-time type of 'index' is (Int32 | Nil)
if index
# In this scope, compile-time type of 'index' is Int32
#my_list[0] # => 2
#my_list[index] # => 6
end
I'm just going to do this:
def get_index(val)
i = 0
while i < #my_list.size
if #my_list[i].value == val
return i
end
i += 1
end
return -1
end
This way only int values will be returned, no nils. It seems to work fine.
better way is to use times method it's more simple and more clear:
def get_index(val)
#my_list.size.times do |i|
return i if #my_list[i] == val
end
-1
end
UPD
or more simple
def get_index(val)
#my_list.index { |value| value == val } || -1
end
I am counting values in each window and find the top values and want to save only the top 10 frequent values of each window to hdfs rather than all the values.
eegStreams(a) = KafkaUtils.createStream(ssc, zkQuorum, group, Map(args(a) -> 1),StorageLevel.MEMORY_AND_DISK_SER).map(_._2)
val counts = eegStreams(a).map(x => (math.round(x.toDouble), 1)).reduceByKeyAndWindow(_ + _, _ - _, Seconds(4), Seconds(4))
val sortedCounts = counts.map(_.swap).transform(rdd => rdd.sortByKey(false)).map(_.swap)
ssc.sparkContext.parallelize(rdd.take(10)).saveAsTextFile("hdfs://ec2-23-21-113-136.compute-1.amazonaws.com:9000/user/hduser/output/" + (a+1))}
//sortedCounts.foreachRDD(rdd =>println("\nTop 10 amplitudes:\n" + rdd.take(10).mkString("\n")))
sortedCounts.map(tuple => "%s,%s".format(tuple._1, tuple._2)).saveAsTextFiles("hdfs://ec2-23-21-113-136.compute-1.amazonaws.com:9000/user/hduser/output/" + (a+1))
I can print top 10 as above (commented).
I have also tried
sortedCounts.foreachRDD{ rdd => ssc.sparkContext.parallelize(rdd.take(10)).saveAsTextFile("hdfs://ec2-23-21-113-136.compute-1.amazonaws.com:9000/user/hduser/output/" + (a+1))}
but I get the following error. My Array is not serializable
15/01/05 17:12:23 ERROR actor.OneForOneStrategy:
org.apache.spark.streaming.StreamingContext
java.io.NotSerializableException:
org.apache.spark.streaming.StreamingContext
Can you try this?
sortedCounts.foreachRDD(rdd => rdd.filterWith(ind => ind)((v, ind) => ind <= 10).saveAsTextFile(...))
Note: I didn't test the snippet...
Your first version should work. Just declare #transient ssc = ... where the Streaming Context is first created.
The second version won't work b/c StreamingContext cannot be serialized in a closure.