Decompressing a netCDF file - compression

I have a comrpessed netCDF file called in.nc with a (_DeflateLevel=4). I want to decompress the file.
I've already tried using the following command but it doesn't change anything:
nccopy -d 0 in.nc out.nc
Here are the in.nc specifications:
time_counter = UNLIMITED ; // (12 currently)
y = 552 ;
x = 552 ;
variables:
float Q16c(time_counter, y, x) ;
Q16c:long_name = "Deep Sediment Particulate organic Matter" ;
Q16c:units = "mg C/m2" ;
Q16c:online_operation = "average" ;
Q16c:interval_operation = "900 s" ;
Q16c:interval_write = "1 month" ;
Q16c:cell_methods = "time: mean (interval: 900 s)" ;
Q16c:_FillValue = 1.e+20f ;
Q16c:missing_value = 1.e+20f ;
Q16c:coordinates = "time_centered nav_lat nav_lon" ;
Q16c:_Storage = "chunked" ;
Q16c:_ChunkSizes = 1, 552, 552 ;
Q16c:_DeflateLevel = 4 ;
Q16c:_Endianness = "little" ;

Related

leetcode 295 median in stream, runtime error?

Leetcode 295 is to find median in a data stream.
I want to use two heaps to implement it. which can make add a data from stream in O(logn), get the percentile in O(1).
left_heap is a min_heap which used to save the left data of requied percentile.
right_heap used to save data which is larger than percentile.
In class SortedStream, which can make add data o(logn) and make findMedian o(1)
#include <iostream>
#include <vector>
#include <climits>
#include <algorithm>
using namespace std;
class SortedStream {
public:
SortedStream(double percent, size_t rsize = 65536*16) : percent_(percent), reserve_size_(rsize) {
init();
}
void push(double v) { // time complexity, o(logn)
++size_;
double left_top = left_data_.back();
if (left_data_.empty() || v <= left_top) { left_data_.push_back(v); std::push_heap(left_data_.begin(), left_data_.end(), std::less<double>{}); }
else { right_data_.push_back(v); std::push_heap(right_data_.begin(), right_data_.end(), std::greater<double>{}); }
size_t idx = size_ * percent_ + 1;
size_t left_size = left_data_.size();
if (idx < left_size) {
// pop left top into right
std::pop_heap(left_data_.begin(), left_data_.end(), std::less<double>{});
double left_top = left_data_.back();
left_data_.pop_back();
right_data_.push_back(left_top);
std::push_heap(right_data_.begin(), right_data_.end(), std::less<double>{});
} else if (idx > left_size) {
// pop right top into left
std::pop_heap(right_data_.begin(), right_data_.end(), std::greater<double>{});
double right_top = right_data_.back();
right_data_.pop_back();
left_data_.push_back(right_top);
std::push_heap(left_data_.begin(), left_data_.end(), std::greater<double>{});
}
}
void init() {
size_t lsize = reserve_size_ * percent_ + 2;
left_data_.reserve(lsize);
right_data_.reserve(reserve_size_ - lsize + 2);
max_ = INT_MIN;
min_ = INT_MAX;
std::make_heap(left_data_.begin(), left_data_.end(), std::less<double>{});
std::make_heap(right_data_.begin(), right_data_.end(), std::greater<double>{});
size_ = 0;
}
size_t size() const { return size_; }
double max() const { return max_; }
double min() const { return min_; }
double percentile() const { // time complexity o(1)
return left_data_.back();
}
public:
double percent_;
size_t size_;
double max_, min_;
std::vector<double> left_data_, right_data_;
size_t reserve_size_;
};
class MedianFinder {
public:
MedianFinder() : ss(0.5){}
void addNum(int num) {ss.push(num);}
double findMedian() {return ss.percentile();}
SortedStream ss;
};
int main() {
MedianFinder* obj = new MedianFinder();
for (size_t i = 0; i< 15; ++i) {
obj->addNum(i);
double param_2 = obj->findMedian();
cout << "i = " << i << " median = " << param_2 << endl;
}
}
it's ok to run in my laptop, but when i submit in leetcode, it comes out:
Line 863: Char 45: runtime error: applying non-zero offset 18446744073709551608 to null pointer (stl_iterator.h)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_iterator.h:868:45
I never see this error.
can you help on this?
I like your (OP's) idea that Heap can be used to solve the task, heap of smaller and larger values. Also as #ArminMontigny suggested one can use std::priority_queue instead of plain heap, because priority queue is based on heap and adds easy to use helper methods. Regular heap is a kind of low-level backend for priority queue.
Based on these two suggestions and inspired by your interesting question I decide to implement short (30 lines) solution for your task (it uses random numbers as example input):
Try it online!
#include <queue>
#include <random>
#include <iostream>
int main() {
std::mt19937_64 rng{123};
std::priority_queue<int> smaller;
std::priority_queue<int, std::vector<int>, std::greater<int>> larger;
for (size_t i = 0; i < 100; ++i) {
int n = rng() % 1000;
if (smaller.empty() || n <= smaller.top())
smaller.push(n);
else
larger.push(n);
while (smaller.size() + 1 < larger.size()) {
smaller.push(larger.top());
larger.pop();
}
while (larger.size() + 1 < smaller.size()) {
larger.push(smaller.top());
smaller.pop();
}
double median = smaller.size() == larger.size() ?
(smaller.top() + larger.top()) / 2.0 :
smaller.size() < larger.size() ? larger.top() : smaller.top();
std::cout << "n = " << n << " med = " << median << " | ";
if ((i + 1) % 4 == 0)
std::cout << std::endl;
}
}
Output:
n = 504 med = 504 | n = 771 med = 637.5 | n = 101 med = 504 | n = 790 med = 637.5 |
n = 380 med = 504 | n = 388 med = 446 | n = 888 med = 504 | n = 406 med = 455 |
n = 53 med = 406 | n = 240 med = 397 | n = 749 med = 406 | n = 438 med = 422 |
n = 566 med = 438 | n = 238 med = 422 | n = 741 med = 438 | n = 817 med = 471 |
n = 810 med = 504 | n = 376 med = 471 | n = 816 med = 504 | n = 503 med = 503.5 |
n = 599 med = 504 | n = 264 med = 503.5 | n = 704 med = 504 | n = 132 med = 503.5 |
n = 740 med = 504 | n = 391 med = 503.5 | n = 563 med = 504 | n = 778 med = 533.5 |
n = 768 med = 563 | n = 136 med = 533.5 | n = 964 med = 563 | n = 368 med = 533.5 |
n = 653 med = 563 | n = 941 med = 564.5 | n = 976 med = 566 | n = 680 med = 582.5 |
n = 546 med = 566 | n = 200 med = 564.5 | n = 387 med = 563 | n = 698 med = 564.5 |
n = 562 med = 563 | n = 251 med = 562.5 | n = 257 med = 562 | n = 735 med = 562.5 |
n = 822 med = 563 | n = 212 med = 562.5 | n = 576 med = 563 | n = 368 med = 562.5 |
n = 783 med = 563 | n = 964 med = 564.5 | n = 234 med = 563 | n = 805 med = 564.5 |
n = 952 med = 566 | n = 162 med = 564.5 | n = 936 med = 566 | n = 493 med = 564.5 |
n = 88 med = 563 | n = 313 med = 562.5 | n = 580 med = 563 | n = 274 med = 562.5 |
n = 353 med = 562 | n = 701 med = 562.5 | n = 882 med = 563 | n = 249 med = 562.5 |
n = 19 med = 562 | n = 482 med = 554 | n = 327 med = 546 | n = 402 med = 525 |
n = 379 med = 504 | n = 521 med = 512.5 | n = 977 med = 521 | n = 550 med = 533.5 |
n = 434 med = 521 | n = 82 med = 512.5 | n = 581 med = 521 | n = 134 med = 512.5 |
n = 532 med = 521 | n = 860 med = 526.5 | n = 562 med = 532 | n = 225 med = 526.5 |
n = 907 med = 532 | n = 837 med = 539 | n = 671 med = 546 | n = 785 med = 548 |
n = 593 med = 550 | n = 533 med = 548 | n = 471 med = 546 | n = 352 med = 539.5 |
n = 388 med = 533 | n = 532 med = 532.5 | n = 310 med = 532 | n = 135 med = 532 |
n = 323 med = 532 | n = 81 med = 526.5 | n = 849 med = 532 | n = 577 med = 532 |
n = 643 med = 532 | n = 956 med = 532.5 | n = 204 med = 532 | n = 383 med = 532 |
Regarding your question about Sanitizer error - this sanitizer is a part of CLang. You can download Clang yourself and try it out on your home laptop, to reproduce exactly same error.
To get same error add option -fsanitize=undefined, when compiling using CLang at home.
For Windows CLang can be downloaded from this page. Also on Windows if you have great package manager Chocolatey, then you can install CLang + LLVM through short command choco install llvm.
For Linux CLang can be installed through sudo apt install clang.
Also you can use great online website GodBolt, by this link, at given link I already chosen CLang for compilation and put necessary options -std=c++11 -O0 -fsanitize=undefined, so you have just to start coding in the window to the left-handside when you open the link.
You have one minor problem
In the line
double left_top = left_data_.back();
At the very beginning, the std::vector "left_data" will be empty. If you try to access the last element of an empty vector, you will get an runtime error.
If you modify this line to for example:
double left_top = left_data_.empty()?0.0:left_data_.back();
Then your program will work as you expect it to work.
I personally find the approach a a little bit too complicated. Maybe you could use a std::multiset or a std::priority_queue. Especially the std::priority_queue will also implement a max-heap and a min-hep for you, without out the overhead of calling std::vectors heap functions.
But I am still in favor of the std::multiset . . .

NCO: Copying data from one netcdf file into another only for a specific area

I have two NetCDF files with the exact same dimensions (time,lat,lon). Below is the header of one of the files.
netcdf file1 {
dimensions:
lon = 360 ;
lat = 177 ;
time = 360 ;
variables:
double lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "Longitude" ;
double lat(lat) ;
lat:units = "degrees_north" ;
lat:long_name = "Latitude" ;
double time(time) ;
time:long_name = "Time" ;
time:units = "months since 1989-05-01 00:00" ;
double tmp(time, lat, lon) ;
tmp:_FillValue = -999000000. ;
}
I would like to copy values from one file into the other, but only for a small region determined by lat1,lat2 and lon1,lon2. Both files have the same time coordinates.
Something like: (lon1<lon<=lon2) & (lat1<lat<=lat2) file1 = file2
I was wondering if I could do that with NCO.
Any help will be appreciated. Thank
Read the manual section on the ncap2 where function. It describes how to use where on a hyperslab. That will do it:
*var_tmp=var2(:,0,:,:);
where (var1 < 0.5) var_tmp=1234;
var2(;,0,:,;)=var_tmp;

Unable to access all the related predicates in Prolog from C++

I am trying to access all the related predicates of a prolog file from C++.
I have the prolog file as,
"new_gryffindor.pl"
sits_right_of(parvati,lavender).
sits_right_of(lavender,neville).
sits_right_of(neville,alicia).
sits_right_of(alicia,fred).
sits_right_of(fred,george).
sits_right_of(george,lee).
sits_right_of(lee,dennis).
sits_right_of(dennis,dean).
sits_right_of(dean,ginny).
sits_right_of(ginny,angelina).
sits_right_of(angelina,seamus).
sits_right_of(seamus,colin).
sits_right_of(colin,harry).
sits_right_of(harry,hermoine).
sits_right_of(hermoine,ron).
sits_right_of(ron,natalie).
sits_right_of(natalie,katie).
sits_right_of(katie,parvati).
sits_left_of(X,Y) :- sits_right_of(Y,X).
are_neighbours_of(X,Y,Z) :- sits_left_of(X,Z),
sits_right_of(Y,Z).
next_to_each_other(X,Y) :- sits_left_of(X,Y);
sits_right_of(X,Y).
And I have integrated C++ and Prolog file with the C++ code,
term_t a = PL_new_term_ref();
term_t b = PL_new_term_ref();
term_t ans = PL_new_term_ref();
PL_put_variable(ans);
predicate_t p_consult = PL_predicate("consult", 1, "database");
term_t t = PL_new_term_ref();
PL_put_string_chars(t, "new_gryffindor.pl");
PL_call_predicate(NULL, 0, p_consult, t);
fun = PL_new_functor(PL_new_atom("sits_right_of"),2);
PL_cons_functor(ans, fun, a, b);
char *fact1;
char *fact2;
if(PL_call(ans, NULL)) {
PL_get_atom_chars(a, &fact1);
PL_get_atom_chars(b, &fact2);
cout << fact1 << " sits right of " << fact2;
}
This C++ Code is giving me the result for the very first predicate of "sits_right_of" ,i.e, as "parvati sits right of lavender". But I want to print all the related predicates of "sits_right_of" in C++ like the prolog which gives next similar predicate's values by using semicolon ";".
sits_right_of(X,Y).
X = parvati,
Y = lavender ;
X = lavender,
Y = neville ;
X = neville,
Y = alicia ;
X = alicia,
Y = fred ;
X = fred,
Y = george ;
X = george,
Y = lee ;
X = lee,
Y = dennis ;
X = dennis,
Y = dean ;
X = dean,
Y = ginny ;
X = ginny,
Y = angelina ;
X = angelina,
Y = seamus ;
X = seamus,
Y = colin ;
X = colin,
Y = harry ;
X = harry,
Y = hermoine ;
X = hermoine,
Y = ron ;
X = ron,
Y = natalie ;
X = natalie,
Y = katie ;
X = katie,
Y = parvati.
I tried looping statements in C++ but it is printing the same output for multiple times..
So please help me out to print the values of other similar predicates of prolog file in C++.

how to do align outcome values in python

i am having trouble with aligning outcome values.
Alist = ["1,25,999",
"123.4,56.7890,13.571",
"1,23.45,6,7.8"]
c = 0
while c < len(Alist):
r = 0
tokens = Alist[c].split(',')
while r < len(Alist[c].split(',')):
if '.' in tokens[r]:
print "%7.2f" %float(tokens[r]), " ",
else :
print "%3d" %float(tokens[r]), " ",
r += 1
print
c += 1
I want to print such as
1 25 999
123.40 56.79 13.57
1 23.45 6. 7.80
but somehow it is printing
1
25
999
123.40
56.79
13.57
1
23.45
6
7.8
and i cannot figure out what is wrong with my coding.
after the r+1, you have a lone print statement. it is at the wrong indention level - move it to the left by 4 spaces (or one tab) and it should work fine.
The print statement should'nt in the 2nd while loop. just:
Alist = ["1,25,999",
"123.4,56.7890,13.571",
"1,23.45,6,7.8"]
c = 0
while c < len(Alist):
r = 0
tokens = Alist[c].split(',')
while r < len(Alist[c].split(',')):
if '.' in tokens[r]:
print "%7.2f" %float(tokens[r]), " ",
else :
print "%3d" %float(tokens[r]), " ",
r += 1
print
c += 1
In [59]: %paste
myList = ["1,25,999",
"123.4,56.7890,13.571",
"1,23.45,6,7.8"]
rows = [r.split(',') for r in myList]
widths = {i:max(len(c) for c in col) for i,col in enumerate(itertools.izip_longest(*rows, fillvalue=""))}
for row in rows:
for i,val in enumerate(row):
print " "*((widths[i] - len(val))/2), val, " "*((widths[i] - len(val))/2) if not (widths[i]-len(val))%2 else " "*((widths[i] - len(val)+1)/2),
print
## -- End pasted text --
1 25 999
123.4 56.7890 13.571
1 23.45 6 7.8

sed prints out all data instead of substituted part which i want to extract [closed]

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I am aware, that this problem has been asked in slightly different ways many times before. But I couldn't adapt those solutions.
I am having a file header.out with the following content:
// some text before this
:type = "Primary Grid: Tau Format" ;
:deformation_timestep = 0. ;
:deformation_alpha = 0. ;
:deformation_dalpha = 0. ;
:deformation_h = 0. ;
:deformation_dh = 0. ;
:deformation_step = 0 ;
:adaptation_level = 0 ;
:connectivity_check = 1 ;
:volume_check = 1 ;
:point_check = 1 ;
:test_lib_check = 1 ;
:periodic_pairs_check = 1 ;
:is_balanced = 0 ;
:preserve_piles = 0 ;
:marker_1 = "WingMidTrailing" ;
:marker_2 = "WingMidLower" ;
:marker_3 = "WingMidUpper" ;
:marker_4 = "WingSide" ;
:marker_5 = "Farfield" ;
:marker_6 = "SymmetryPlane" ;
:marker_7 = "WingInnerLower" ;
:marker_8 = "WingInnerUpper" ;
:marker_9 = "WingInnerTrailing" ;
:marker_10 = "WingOuterLower" ;
:marker_11 = "WingOuterUpper" ;
:marker_12 = "WingOuterTrailing" ;
}
I need to extract the ID of the marker that belongs to the string "SymmetryPlane"
So in this case i want to have 6 as the output. All tries using sed and regular expressions failed.
What i have tried is:
sed 's/.*marker_\(.*\) = "SymmetryPlane.*/\1/' header.out
But this only gives me
:type = "Primary Grid: Tau Format" ;
:deformation_timestep = 0. ;
:deformation_alpha = 0. ;
:deformation_dalpha = 0. ;
:deformation_h = 0. ;
:deformation_dh = 0. ;
:deformation_step = 0 ;
:adaptation_level = 0 ;
:connectivity_check = 1 ;
:volume_check = 1 ;
:point_check = 1 ;
:test_lib_check = 1 ;
:periodic_pairs_check = 1 ;
:is_balanced = 0 ;
:preserve_piles = 0 ;
:marker_1 = "WingMidTrailing" ;
:marker_2 = "WingMidLower" ;
:marker_3 = "WingMidUpper" ;
:marker_4 = "WingSide" ;
:marker_5 = "Farfield" ;
6
:marker_7 = "WingInnerLower" ;
:marker_8 = "WingInnerUpper" ;
:marker_9 = "WingInnerTrailing" ;
:marker_10 = "WingOuterLower" ;
:marker_11 = "WingOuterUpper" ;
:marker_12 = "WingOuterTrailing" ;
}
I am not sure why this is happening.
Thanks in advance!
Just suppress the output (-n) and print after substitution (p):
sed -n 's/.*marker_\(.*\) = "SymmetryPlane.*/\1/p' header.out