I use SEAL 2.3.1 and this is my parameter setting:
seal::EncryptionParameters parms;
parms.set_poly_modulus("1x^2048 + 1"); // n = 2048
parms.set_coeff_modulus(coeff_modulus_128(2048)); // q = 54-bit prime
parms.set_plain_modulus(1 << 8); // t = 256
seal::SEALContext context(parms);
And some Ciphertext encrypted1; holding the number 5. The manual say that one can use the seal::Simulator class for reading the noise budget without the secret key. The only thing that I've found was this in the simulator.h file.
/**
Creates a simulation of a ciphertext encrypted with the specified encryption
parameters and given invariant noise budget. The given noise budget must be
at least zero, and at most the significant bit count of the coefficient
modulus minus two.
#param[in] parms The encryption parameters
#param[in] noise_budget The invariant noise budget of the created ciphertext
#param[in] ciphertext_size The size of the created ciphertext
#throws std::invalid_argument if ciphertext_size is less than 2
#throws std::invalid_argument if noise_budget is not in the valid range
*/
Simulation(const EncryptionParameters &parms, int ciphertext_size,
int noise_budget);
I can set it with some other Ciphertext encrypted2:
seal::Simulation(parms, encrypted2.size(), (context.total_coeff_modulus().significant_bit_count() - log2(context.poly_modulus().coeff_count() - 1) - log2(context.plain_modulus().value()));
But using this will only create a simulated Ciphertext without any real connection to the actual encrypted1 Ciphertext noise budget.
Is there a way to approximate the noise budget of encrypted1 without the secret key? This situations is important when I or someone else does some computation on externally stored Ciphertexts, e.g. in a cloud database and needs to check the noise budget server side without revealing the secret key.
The Simulation class is meant to estimate the noise budget consumption in various operations so that those operations don't actually have to be executed on real data. Moreover, it uses heuristic upper bounds estimate for the noise consumption, i.e. most likely it overestimates the noise consumption and this effect becomes more pronounced when the computation is more complicated, sometimes resulting in huge overestimates of the noise consumption. Of course, the idea is that the computation is guaranteed to work if it works according to the simulator. A typical use of Simulation would be through the ChooserPoly (and related) classes; this is demonstrated in one of the examples in SEALExamples/main.cpp for SEAL versions < 3.0.
It is impossible to know or estimate the noise in a ciphertext without knowing how that ciphertext was produced. So if I give you a ciphertext without telling you anything else (except encryption parameter), then you should not be able to know anything about the noise budget unless you know the secret key. I agree that in some cases it could be important for someone to know right away if a ciphertext is still valid for further computations, but it's not possible without some external mechanism.
Related
I am reading the following article about PoS
http://earlz.net/view/2017/07/27/1904/the-missing-explanation-of-proof-of-stake-version
In this article, the author says
The block hash must be signed by the public key in the staking transaction's second vout
Why is this necessary? When using PoS, because the coin stake transaction's input comes from the miner, so the miner already provides an unlocking script on the inputs of the coin stake transaction. Why does she need to sign the block?
A reference implementation can be found in PIVX:
class CBlock : public CBlockHeader
{
public:
// network and disk
std::vector<CTransaction> vtx;
// ppcoin: block signature - signed by one of the coin base txout[N]'s owner
std::vector<unsigned char> vchBlockSig;
where vchBlockSig stores the signature which is signed by
key.Sign(block.GetHash(), block.vchBlockSig)
In the PoW systems, block signing is not needed, because of block ID id generating by hashing root of Merkle Tree (Merkle Root) of payload transactions and nonce, until hash become less than target.
If do analogous approach in PoS system, them malicious minter can generate lot of attempts with different output hashes from a same kernel UTXO (transaction output, which mints coins) - just by modify nonce and/or rearrange transaction in Merkle Tree, there is lot of combinations. And by this way, he can reduce PoS to PoW (lot of hashing attempts with same data).
To prevent such degradation, PoS cryptos (PPC, EMC, etc) limits number of attempts for some specific UTXO. And hash result (which compared to a target) depends only from kernel UTXO and from current time, and independent from nonce, block payload, and so. As result, PoS minter can make a single attempt for each matured UTXO only once per second.
But, with this approach, block content is not participate in the kernel hash, which compares to target.
As result, if minter does not sign a block, malicious actor can do following attack: He can intercept from the network a freshly-minted block from a minter, modify payload transactions and Merkle Tree and block hash (for example, add double spend TX), and redistribute the modified block over the network. Such block will contains valid coinstake transaction (which spends kernel UTXO), and will be accepted by network nodes.
To prevent this "modify freshly-minted block on the fly", block is signed by address of kernel UTXO. By this signature, minter provides a proof: Block is created by same minter, who generates a coinstake TX.
Thus, with PoS, block generation is following:
Find appropriate kernel UTXO
Generate coinstake transaction, which sends coins from kernel UTXO address to itself.
Create a new block,contains this coinstake TX and payload TXes
Sign this block with coin address of kernel UTXO.
Practically, enough to sign just a header, contains Merkle Root.
I'm trying to play with Kafka Stream to aggregate some attribute of People.
I have a kafka stream test like this :
new ConsumerRecordFactory[Array[Byte], Character]("input", new ByteArraySerializer(), new CharacterSerializer())
var i = 0
while (i != 5) {
testDriver.pipeInput(
factory.create("input",
Character(123,12), 15*10000L))
i+=1;
}
val output = testDriver.readOutput....
I'm trying to group the value by key like this :
streamBuilder.stream[Array[Byte], Character](inputKafkaTopic)
.filter((key, _) => key == null )
.mapValues(character=> PersonInfos(character.id, character.id2, character.age) // case class
.groupBy((_, value) => CharacterInfos(value.id, value.id2) // case class)
.count().toStream.print(Printed.toSysOut[CharacterInfos, Long])
When i'm running the code, I got this :
[KTABLE-TOSTREAM-0000000012]: CharacterInfos(123,12), 1
[KTABLE-TOSTREAM-0000000012]: CharacterInfos(123,12), 2
[KTABLE-TOSTREAM-0000000012]: CharacterInfos(123,12), 3
[KTABLE-TOSTREAM-0000000012]: CharacterInfos(123,12), 4
[KTABLE-TOSTREAM-0000000012]: CharacterInfos(123,12), 5
Why i'm getting 5 rows instead of just one line with CharacterInfos and the count ?
Doesn't groupBy just change the key ?
If you use the TopologyTestDriver caching is effectively disabled and thus, every input record will always produce an output record. This is by design, because caching implies non-deterministic behavior what makes itsvery hard to write an actual unit test.
If you deploy the code in a real application, the behavior will be different and caching will reduce the output load -- which intermediate results you will get, is not defined (ie, non-deterministic); compare Michael Noll's answer.
For your unit test, it should actually not really matter, and you can either test for all output records (ie, all intermediate results), or put all output records into a key-value Map and only test for the last emitted record per key (if you don't care about the intermediate results) in the test.
Furthermore, you could use suppress() operator to get fine grained control over what output messages you get. suppress()—in contrast to caching—is fully deterministic and thus writing a unit test works well. However, note that suppress() is event-time driven, and thus, if you stop sending new records, time does not advance and suppress() does not emit data. For unit testing, this is important to consider, because you might need to send some additional "dummy" data to trigger the output you actually want to test for. For more details on suppress() check out this blog post: https://www.confluent.io/blog/kafka-streams-take-on-watermarks-and-triggers
Update: I didn't spot the line in the example code that refers to the TopologyTestDriver in Kafka Streams. My answer below is for the 'normal' KStreams application behavior, whereas the TopologyTestDriver behaves differently. See the answer by Matthias J. Sax for the latter.
This is expected behavior. Somewhat simplified, Kafka Streams emits by default a new output record as soon as a new input record was received.
When you are aggregating (here: counting) the input data, then the aggregation result will be updated (and thus a new output record produced) as soon as new input was received for the aggregation.
input record 1 ---> new output record with count=1
input record 2 ---> new output record with count=2
...
input record 5 ---> new output record with count=5
What to do about it: You can reduce the number of 'intermediate' outputs through configuring the size of the so-called record caches as well as the setting of the commit.interval.ms parameter. See Memory Management. However, how much reduction you will be seeing depends not only on these settings but also on the characteristics of your input data, and because of that the extent of the reduction may also vary over time (think: could be 90% in the first hour of data, 76% in the second hour of data, etc.). That is, the reduction process is deterministic but from the resulting reduction amount is difficult to predict from the outside.
Note: When doing windowed aggregations (like windowed counts) you can also use the Suppress() API so that the number of intermediate updates is not only reduced, but there will only ever be a single output per window. However, in your use case/code you the aggregation is not windowed, so cannot use the Suppress API.
To help you understand why the setup is this way: You must keep in mind that a streaming system generally operates on unbounded streams of data, which means the system doesn't know 'when it has received all the input data'. So even the term 'intermediate outputs' is actually misleading: at the time the second input record was received, for example, the system believes that the result of the (non-windowed) aggregation is '2' -- its the correct result to the best of its knowledge at this point in time. It cannot predict whether (or when) another input record might arrive.
For windowed aggregations (where Suppress is supported) this is a bit easier, because the window size defines a boundary for the input data of a given window. Here, the Suppress() API allows you to make a trade-off decision between better latency but with multiple outputs per window (default behavior, Suppress disabled) and longer latency but you'll get only a single output per window (Suppress enabled). In the latter case, if you have 1h windows, you will not see any output for a given window until 1h later, so to speak. For some use cases this is acceptable, for others it is not.
I'm writing Data Encryption Standard "cracker" using C++ and CUDA. It was going to be simple brute-force - trying all possible keys to decrypt encrypted data and check if result is equal to initial plain-text message.
The problem is that generation of 2^56 keys takes time (and memory). My first approach was to generate keys recursively and save them to file.
Do you have any suggestions how to improve this?
You don' really need recursion, neither you need storing your keys.
All space of DES keys (if we don't count 12 or so weak keys, which won't change anything for your purposes) is a space of 56-bit-long numbers (which BTW fit into standard uint64_t), and you can just iterate through numbers from 0 to 2^56-1, feeding the next number as a 56-bit number to your CUDA core whenever the core reports that it is done with the previous key.
If not for cores, the code could look such as:
for(uint64_t i=0;i<0xFFFFFFFFFFFFFFULL /* double-check number of F's so the number is 2^56-1 */;++i) {
uint8_t key[7];
//below is endianness-agnostic conversion
key[0] = (uint8_t)i;
key[1] = (uint8_t)(i>>8);
key[2] = (uint8_t)(i>>16);
key[3] = (uint8_t)(i>>24);
key[4] = (uint8_t)(i>>32);
key[5] = (uint8_t)(i>>40);
key[6] = (uint8_t)(i>>48);
bool found = try_your_des_code(key,data_to_decrypt);
if(found) printf("Eureka!\n");
}
To allow restarting your program in case if anything goes wrong, you need to store (in persistent storage, such as file) only this number i (with cores, strictly speaking - the number i should be written to persistent storage only after all the numbers before it has already been processed by CUDA cores, but generally the difference of 2000 or so keys won't make any difference performance-wise).
I an new to overtone/supercollider. I know how sound forms physically. However I don't understand the magic inside overtone's sound generating functions.
Let's say I have a basic sound:
(definst sin-wave [freq 440 attack 0.01 sustain 0.4 release 0.1 vol 0.4]
(* (env-gen (lin-env attack sustain release) 1 1 0 1 FREE)
(+ (sin-osc freq)
(sin-osc (* freq 2))
(sin-osc (* freq 4)))
vol))
I understand the ASR cycle of sound envelope, sin wave, frequency, volume here. They describe the amplitude of the sound over time. What I don't understand is the time. Since time is absent from the input of all functions here, how do I control stuffs like echo and other cool effects into the thing?
If I am to write my own sin-osc function, how do I specify the amplitude of my sound at specific time point? Let's say my sin-osc has to set that at 1/4 of the cycle the output reaches the peak of amplitude 1.0, what is the interface that I can code with to control it?
Without knowing this, all sound synth generators in overtone doesn't make sense to me and they look like strange functions with unknown side-effects.
Overtone does not specify the individual samples or shapes over time for each signal, it is really just an interface to the supercollider server (which defines a protocol for interaction, of which the supercollider language is the canonical client to this server, and overtone is another). For that reason, all overtone is doing behind the scenes is sending signals for how to construct a synth graph to the supercollider server. The supercollider server is the thing that is actually calculating what samples get sent to the dac, based on the definitions of the synths that are playing at any given time. That is why you are given primitive synth elements like sine oscillators and square waves and filters: these are invoked on the server to actually calculate the samples.
I got an answer from droidcore at #supercollider/Freenode IRC
d: time is really like wallclock time, it's just going by
d: the ugen knows how long each sample takes in terms of milliseconds, so it knows how much to advance its notion of time
d: so in an adsr, when you say you want an attack time of 1.0 seconds, it knows that it needs to take 44100 samples (say) to get there
d: the sampling rate is fixed and is global. it's set when you start the synthesis process
d: yeah well that's like doing a lookup in a sine wave table
d: they'll just repeatedly look up the next value in a table that
represents one cycle of the wave, and then just circle around to
the beginning when they get to the end
d: you can't really do sample-by sample logic from the SC side
d: Chuck will do that, though, if you want to experiment with it
d: time is global and it's implicit it's available to all the oscillators all the time
but internally it's not really like it's a closed form, where you say "give me the sample for this time value"
d: you say "time has advanced 5 microseconds. give me the new value"
d: it's more like a stream
d: you don't need to have random access to the oscillators values, just the next one in time sequence
I'm trying to create a standalone AGC using WebRtc library. (Input - wav file, output - wav file with adjusted gain). But at this time I have some problems with this issue.
I'm trying to use functions which are declared in gain_control.h file. When I'm using WebRtcAgc_Process(....) I obtain constant gain, which applies to whole signal, but not nonlinear gain which depends from input signal magnitude.
May be I should use another functions for my purpose? How can I implement AGC via WebRTC library?
The AGC's main purpose is to provide a recommended system mic volume which the user is expected to set through the OS. If you would like to apply a purely digital gain, you can configure it in one of two modes (from modules/audio_processing/include/audio_processing.h, but gain_control.h has analogous modes):
// Adaptive mode intended for situations in which an analog volume control
// is unavailable. It operates in a similar fashion to the adaptive analog
// mode, but with scaling instead applied in the digital domain. As with
// the analog mode, it additionally uses a digital compression stage.
kAdaptiveDigital,
// Fixed mode which enables only the digital compression stage also used by
// the two adaptive modes.
//
// It is distinguished from the adaptive modes by considering only a
// short time-window of the input signal. It applies a fixed gain through
// most of the input level range, and compresses (gradually reduces gain
// with increasing level) the input signal at higher levels. This mode is
// preferred on embedded devices where the capture signal level is
// predictable, so that a known gain can be applied.
kFixedDigital
You can set these through WebRtcAgc_Init(), though unless you need to avoid the overhead, I'd recommend just using the AudioProcessing class.
refer to http://osxr.org/android/source/external/webrtc/src/modules/audio_processing/agc/interface/gain_control.h#0133
The gain adjustments are done only during 0135 * active periods of
speech. The input speech length can be either 10ms or 0136 * 20ms and
the output is of the same length.
quick overview of webrtcage_process
int WebRtcAgc_Process(void* agcInst,
const WebRtc_Word16* inNear,
const WebRtc_Word16* inNear_H,
WebRtc_Word16 samples,
WebRtc_Word16* out,
WebRtc_Word16* out_H,
WebRtc_Word32 inMicLevel,
WebRtc_Word32* outMicLevel,
WebRtc_Word16 echo,
WebRtc_UWord8* saturationWarning);