I am trying to to set a value for all items in a domain that do not already have a certain value and have an additional flag set.
Basically for all my items,
SET ValueA to 100 if ValueB is 0
But I am confused about how to achieve this. So far ive been setting the value for individual items by just using a PutRequest like this:
ArrayList<ReplaceableAttribute> newAttributes = new ArrayList<ReplaceableAttribute>();
newAttributes.add(new ReplaceableAttribute("ValueA",Integer.toString(100), true));
PutAttributesRequest newRequest = new PutAttributesRequest();
newRequest.setDomainName(usersDomain);
newRequest.setItemName(userID);
newRequest.setAttributes(newAttributes);
sdb.putAttributes(newRequest);
This works for an individual item and requires me to first get the item name (userID). Does this means that I have to "list" all of my items and do this 1 by 1?
I suppose that since I have around 19000+ items I would also have to use the token to get the next set after the 2000 limit right?
Isn't there a more efficient way? This might not be so heavy right now but I expect to eventually have over 100k items.
PD: I am using the AWS Java SDK for Eclipse.
If you are talking about how you can do it grammatically by writing your own code then Yes. First you have to know all item name i.e in your case UserID and then you need to set a value one by one. You can use BatchPUTAttribute in this case. Using Batch PUT you can update 25 items in one request. You can do 5 to 20 BatchPutAttribute requests in parallel threads. Know more to tune the performance.
If you need to do it somehow in tricky way then you can use SDBExplorer. Please Remember it will set 100 for all items because SDBExplorer does not support conditional PUT. If you would like to set it anyway then Follow these steps-
Download SDBExplorer zip version form download page.
Extract it and run the executable.
Download 30 days trial license.
Once license has been downloaded main UI will open.
Provide valid Access Key and Secret keys and click on "GO" button.
You will see list of domains in Left side tree.
Right click on the domain in which you would like to set value for all item.
Choose "Export to CSV" option.
Export the content of domain into CSV. http://www.sdbexplorer.com/documentation/simpledb--how-to-export-domain-in-csv-using-sdbexplorer.html
Go to path where your domain has exported.
Open CSV file.
Your first column is item name.
Delete all columns other then item Name and column "ValueA".
Set 100 for all item name under "ValueA" column.
Save the CSV.
Go to the SDBExplorer main UI.
Select the same domain.
Click on "Import" option from tool bar.
A panel will open.
Now Import the data into the Domain. http://www.sdbexplorer.com/documentation/simpledb--how-to-upload-csv-file-data-and-specifying-column-as-amazon-simple-db-item-name.html
Once import is done, explore the domain and you will find the value 100 set to all items for column ValueA.
Please try the steps first on any dummy domain.
What exactly I am trying to suggest you?
To know all item name in your domain, I am suggesting you to export all content of your domain into CSV file at local file system. Once you get all item name in CSV, keep only one column "ValueA". Set "100" for all the items in CSV file and upload/import the content back into domain.
Discloser: I am one of the developer of SDBExplorer.
Related
I want to automate a few search in one, here are the steps:
Search in Kibana for this ID:"b2c729b5-6440-4829-8562-abd81991e2a0" which will return me a bunch of logs. Of these logs I need to take the first and the last timestamp:
I now would like to store these two data FROM: September 3rd 2019, 21:28:22.155, TO: September 3rd 2019, 21:28:23.524 in 2 variables
Run a second search in Kibana for the word "fail" in between these two variable of time
How to automate the whole process without need of copy/paste and running a second query?
EDIT:
SHORT STORY LONG: I work in a company that produce a software for autonomous vehicles.
SCENARIO: A booking is rejected and we need to understand why.
WHERE IS THE PROBLE: I need to monitor just a few seconds of logs on 3 different machines. Each log is completely separated, there is no relation between the logs so I cannot write a query in discover, I need to run 3 separated queries.
EXAMPLE:
A booking was rejected, so I open Chrome and I search on "elk-prod.myhost.com" for the BookingID:"b2c729b5-6440-4829-8562-abd81991e2a0" and I have a dozen of logs returned during a range of 2 seconds (FROM: September 3rd 2019, 21:28:22.155, TO: September 3rd 2019, 21:28:23.524).
Now I need to know what was happening on the car so I open a new Chrome tab and I search on "elk-prod.myhost.com" for the CarID: "Tesla-45-OU" on the time range FROM: September 3rd 2019, 21:28:22.155, TO: September 3rd 2019, 21:28:23.524
Now I need to know why the server which calculate the matching rejected the booking so I open a new Chrome tab and I search for the word CalculationMatrix always on the time range FROM: September 3rd 2019, 21:28:22.155, TO: September 3rd 2019, 21:28:23.524
CONCLUSION: I want to stop to keep opening Chrome tabs by hand and automate the whole thing. I have no idea around what time the book was made so I first need to search for the BookingID "b2c729b5-6440-4829-8562-abd81991e2a0", then store the timestamp of first and last log and run a second and third query based on those timestamps.
There is no relation between the 3 logs I search so there is no way to filter from the Discover, I need to automate 3 different query.
Here is how I would do it. First of all, from what I understand, you have three different indexes:
one for "bookings"
one for "cars"
one for "matchings"
First, in Discover, I would create three Saved Searches, one per index pattern. Then in Visualize, I would create a Vertical bar chart on the bookings saved search (Bucket X-Axis by date_histogram on the timestamp field, leave the rest as is). You'll get a nice histogram of all your booking events bucketed by time.
Finally, I would create a dashboard and add the vertical bar chart + those three saved searches inside it.
When done, the way I would search according to the process you've described above is as follows:
Search for the booking ID b2c729b5-6440-4829-8562-abd81991e2a0 in the top filter bar. In the bar chart histogram (bookings), you will see all documents related to the selected booking. On that chart, you can select the exact period from when the very first booking document happened to the very last. This will adapt the main time picker at the top and the start/end time will be "remembered" by Kibana
Remove the booking ID from the top filter (since we now know the time range and Kibana stores it). Search for Tesla-45-OU in the top filter bar. The bar histogram + the booking saved search + the matchings saved search will be empty, but you'll have data inside the second list, the one for cars. Find whatever you need to find in there and go to the next step.
Remove the car ID from the top filter and search for ComputationMatrix. Now the third saved search is going to show you whatever documents you need to see within that time range.
I'm lacking realistic data to try this out, but I definitely think this is possible as I've laid out above, probably with some adaptations.
Kibana does work like this (any order is ok):
Select time filter: https://www.elastic.co/guide/en/kibana/current/set-time-filter.html
Add additional criteria for search like for example field s is b2c729b5-6440-4829-8562-abd81991e2a0.
Add aditional criteria for search like for example field x is Fail.
Additionaly you can view surrounding documents https://www.elastic.co/guide/en/kibana/current/document-context.html#document-context
This is how Kibana works.
You can prepare some filters beforehands, save them and then use them if you want to automate the process of discovering somehow.
You can do that in Discover tab in Kibana using New/Save/Open options.
Edit:
I do not think you can achieve what you need in Kibana. As I mentioned earlier one option is to change the data that is comming to Elasticsearch so you can search for it via discover in Kibana. Another option could be builiding for example Java application, that is using Elasticsearch - then you can write algorithm that returns the data that you want. But i think it's a big overhead and I recommend checking the data first.
Edit: To clarify - you can create external Java let's say SpringBoot application that uses Elasticsearch - all the data that you need is inside it.
But in this option you will not use Kibana at all.
You can export the result to csv or what you want in the code.
SpringBoot application can ask ElasticSearch for whatever it needs, then it would be easy to store these time variables inside of Java code.
EDIT: After OP edited question to change it dramatically:
#FrancescoMantovani Well the edited version is very different from where you first posted here How to automate the whole process without need of copy/paste and running a second query? and search for word fail in a single shot. In accepted answer you are still using a three filters one at a time so it is not one search, but three.
What's more if you would use one index, and send data from multiple hosts via filebeat you don't even to have to create this dashboard to do that. Then you can you can select the exact period from when the very first document happened to the very last regarding filter and then remove it and add another filter that you need - it's simple as that. Before you were writing about one query,
How to automate the whole process without need of copy/paste and
running a second query?
not three. And you don't need to open new tab in Chrome each time you want to change filter just organize the data by for example using filebeat as mentioned before.
There is no relation between the 3 logs
From what you wrote the realation exist and it is time.
If the data is in for example three diferent indicies (cause documents don't have much similiar data) you can do it like that:
You change them easily in dicover see:
You can go to discover select index 1 search, select time range that you need, when you change index the time range is still the one you selected, you only need to change filter - you will get what you need.
I'v been asked to create an app in Oracle Apex that will allow me to drop a CSV file. The file contains a list of all active physicians and associated info in my area. I do not know where to begin! Requirements:
-after dropping CSV file to apex, remove unnecessary columns
-edit data in each field, ie if phone# > 7 characters and begins with 1, remove 1. Or remove all special characters from a column.
-The CSV contains physicians of every specialty, I only want to upload specific specialties to the database table.
I have a small amount of SQL experience from Uni, and I know some HTML and CSS, but beyond that I am lost. Please help!
Began tutorial on Oracle-Apex. Created upload wizard on a dev environment
User drops CSV file to apex
Apex edits columns to remove unneccesary characteres
Only uploads specific columns from CSV file
Only adds data when column "Specialties" = specific specialties
Does not add redundant data (physician is already located in table, do nothing)
Produces report showing all new physicians added to table
Huh, you're in deep trouble as you have to do some job using a tool you don't know at all, with limited knowledge of SQL language. Yes, it is said that Apex is simple to use, but nonetheless ... you have to know at least something. Otherwise, as you said, you're lost.
See if the following helps.
there's the CSV file
create a table in your database; its description should match the CSV file. Mention all columns it contains. Pay attention to datatypes, column lengths and such
this table will be "temporary" - you'll use it every day to load data from CSV files: first you'll delete all it contains, then load new rows
using Apex "Create page" Wizard, create the "Data loading" process. Follow the instructions (and/or read documentation about it). Once you're done, you'll have 4 new pages in your Apex application
when you run it, you should be able to load CSV file into that temporary table
That's the first stage - successfully load data into the database. Now, the second stage: fix what's wrong.
create another table in the database; it will be the "target" table and is supposed to contain only data you need (i.e. the subset of the temporary table). If such a table already exists, you don't have to create a new one.
create a stored procedure. It will read data from the temporary table and edit everything you've mentioned (remove special characters, remove leading "1", ...)
as you have to skip physicians that already exist in the target table, use NOT IN or NOT EXISTS
then insert "clean" data into the target table
That stored procedure will be executed after the Apex loading process is done; a simple way to do that is to create a button on the last page which will - when pressed - call the procedure.
The final stage is the report:
as you have to show new physicians, consider adding a column (into the target table) which will be a timestamp (perhaps DATE is enough, if you'll be doing it once a day) or process_id (all rows inserted in the same process will share the same value) so that you could distinguish newly added rows from the old ones
the report itself would be an Interactive report. Why? Because it is easy to create and lets you (or end users) to adjust it according to their needs (filter data, sort rows in a different manner, ...)
Good luck! You'll need it.
I’m not sure where to start on this project. I know how to read the contents of the excel spreadsheet, I know how to identify the header row, I know how to loop over the contents. I believe I have the UX portion worked out but I am not sure how to process the data.
I’ve googled and only found .Net solutions but I’m looking for a ColdFusion/Lucee solution.
I have a working form allowing me to map a user's spreasheet column to my database values (this is being kept simple for this post; user does not have direct access to the database).
Now that I have my data, I'm not sure how to loop over the data results. I believe there will be several loops (an outer and an inner). Then of course I also need to loop over the file contents but I think if I can get the headings mapped out,I can figure out the remaining.
Any good links, tutorials, or guides would be greatly appreciated.
Some pseudo code might be enough to get me started.
User uploads form
System reads headers and content.
User is presented form with a list of columns from their uploaded spreadsheet to match with available database fields (eg “column1” matches “customer name”.
User submits form.
Now what?
UPDATED
Here is what the data looks like AFTER the mapping has been done in my form. The column deliiter is the ::: and within the column the ||| indicates the ID associated with the selected column value. I've included the id and the column value since I plan on displaying the mapping again as a confirmation. Having the ID saves a trip to the database.
If I understand correctly, your question is: how do you provide the user a form allowing them to map their spreadsheet columns to that of the database
Since you have their spreadsheet column names, and you have the database column names, then this problem is essentially a UI/UX problem. You need to show both lists, and allow the user to map them. I can imagine several approaches to this. My first thought would be some sort of drag/drop operation, as follows:
Create a list of boxes, one for each field in your database table, and include the field name in (or above) the box. I'll call this the db field list. Then, create another list for each column from the spreadsheet, which I'll call the spreadsheet column list. The user would drag/drop items from the spreadsheet column list to the db field list.
When a mapping has been completed by the user, you would store the column/field names in as data for the DOM element of the db field list box. Then upon submission, you would acquire the mapping data by visiting each box and adding it to an array. Then you would serialize that array into JSON and send that to your form submission handler.
This could be difficult or easy, depending on your knowledge of UI implementations using JavaScript. jQuery makes this easy (if you know jQuery). There's even a jquery UI plugin that does this: https://jqueryui.com/droppable/.
A quick search for javascript drag drop would help, and here's a few articles I found:
https://www.w3schools.com/html/html5_draganddrop.asp
https://medium.com/quick-code/simple-javascript-drag-drop-d044d8c5bed5
You would also need to submit the array of mappings using javascript. You could search for that as well, and here's an article I found:
https://codereview.stackexchange.com/questions/94493/submit-an-array-as-an-html-form-value-using-javascript
I'm developing an application in Oracle Application Express (APEX).
First page contains list of projects as a tabular report.
Clicking any of the rows forwards to the next page, where records can be edited. I've implemented it with following settings:
Link column: link to custom target
Target: Page in this application
Until this is fine.
My problem is how to pass actual report to the next page?
My table, which is the basis of the report has primary key (ID), and also owner & title combination is unique. Currently ID column is not included in the report.
Also the second page doesn't currently contain field showing ID, as this information isn't important to the users.
I know I could set ID column in report, and create a read only (even hidden) text box in the next page, however I'm looking for a more elegant solution. What is the standard way to solve this?
I wonder if you are asking: "How do I pass a value from page 1 to page 2 so page 2 can use the value to do a query and present the results. If so, here is how it's done.
On page P1, the report, for example, select the attributes for the report region under the region in the Rendering pane on the left of the page designed.
Under Attribute Properties on the right side, look for Link Column and set it to "Link to custom target". Then click Target.
Select the page and then in the Set Items section, on the left, under name, select the PK ID field to receive the passed value ex: P2_ID. On the right under Value select the field to pass the value, ex: #P1_ID# and click ok.
Now, when the link on page 1 report is clicked, the P1_ID is saved into Session state by Apex and passed to P2 which then performs a FETCH using the passed value.
You can read more about Session State here. Also, be aware there are security settings which affect what params can and can't be passed in the URL.
Clicking "Session" in the developer toolbar will enable you to see the session variables being passed.
If you mean "How do I store values in the app that can be accessed anywhere in the app - like a global variable" Then look at Application Items.
As always, please include version numbers in these posts.
When you create a target page let's say Page 3
And you create some items, let's say P3_ITEM_1,P3_ITEM_2, etc
You can assign values to each of them through the url in the original page
The complete APEX URL Syntax looks like this:
http://apex.oracle.com/pls/apex/f?p=AppId:PageId:Session:Request:Debug:ClearCache:Params:ParamValues:PrinterFriendly
Let’s take a closer look:
http:// – the protocol, can be http or https
apex.oracle.com – your domain/host/server, whatever you want to call it. Can also be localhost.
/pls – indicates that you are using Oracle HTTP Server with mod_plsql. If you are using APEX Listener or Embedded PL/SQL Gateway this part is obsolete/missing.
/apex – the entry from your dads.conf file (this a file on your application-server or EPG where the target database is configured) – in case of EPG its just one entry pointing to localhost, in case of an OAS you can have multiple entries, each pointing to an other database
/f?p= – procedure “f” is called and parameter “p” is set to the complete rest of the string. Remember: APEX uses mod_plsql. “f” is a public procedure, this is the main entrypoint for APEX. Or you could say: “f” is APEX.
AppId – the number or the Alias of the Application
:PageId – the number or the Alias of the Page
:Session – unique Session ID, can be 0 for Public Pages or empty (then APEX creates a new Session)
:Request – a Request Keyword. This is basically free text, just a string you can specify to react in a process or region condition on. e.g. you could pass the keyword “CREATE” and have a condition on the delete button of your page saying “dont’t display this button if request is CREATE”.
In other words: use the REQUEST to control the behaviour of your page.
When pressing a button, the button sets the REQUEST to the button-value (e.g. SAVE), so that you can control the processes in the page processing (Submit) phase.
:Debug – set to YES (uppercase!) switches on the Debug-Mode which renders debug-messages and timestamps in your Browser window. This helps to detect wrong behaviour of your page or performance issues and everything else. Every other value then YES turns the Debug-Mode off
:ClearCache – you can put a page id or a list of page ids here (comma-separated) to clear the cache for these pages (set session state to null, …). But there is more: RP resets the pagination of reports on the page(s), a collection name deletes the collection, APP clears all pages and application-items, SESSION does the same as APP but for all applications the session-id has been used in.
:Parameters – comma seperated list of page-item names. Good practice is to set only those page-items which are on the page you are going to. Accepts page-items as well as application-items.
:ParamValues – comma separated list of values. Each value is assigned to the corresponding Parameter provided in ParamNameList (first value assigned to first parameter, second value assigned to second parameter, and so on…).
The trick here is not having values which contain either a comma “,” or a colon “:”. Both would lead to side-effects and errors, as APEX gets confused when parsing the URL. Using a comma works, if enclosed by slashes: e.g. \123,89.
:PrinterFriendly – set to YES (uppercase!) switches the page into PrinterFriendly-Mode, uses the Printerfriendly template to render the Page. You can also hide regions or other elements in PrinterFriendly-Mode using the PRINTER_FRIENDLY variable in a condition.
In your case you'd use Params:ParamValues like this:
P3_ITEM_1,P3_ITEM_2:someValue_1,someValue_2
Documentation
I need to copy selected row values and store as a new record.
I am using Oracle Apex 4.2 and Tabular Form.
I need to use checkbox to select the rows and button copy. When i select multiple rows followed by click copy button to copy all the selected row values as new rows and save.
Can anyone Help
Copying Records Through an APEX Tabular Form Input
The idea of cloning existing records from a single table through an Oracle APEX Tabular Form works without much interference with the default design that you can set up through the APEX wizard for page region content.
Build a table with an independent primary key.
Suggested to include two auxiliary columns: COPY_REQUEST and COPIED_FROM for running copy operations. Specific form elements will map to these columns on the tabular form that will be set up.
Build an Oracle stored procedure that can read which records need to be copied. This procedure will be invoked each time the SUBMIT button is pressed.
(optional) Consider including a suppression of step (3) in the event that there is nothing to process (i.e., no records marked for copying).
The Working Table for Receiving Input: COPY_ME
TIP: You will have an easier time if you use the standard TABLE creation wizard. Designate CUSTOMER_ID as the PRIMARY_KEY and have APEX create its standard auto-incrementing functionality on top. (sequence plus trigger set up.)
Here's the sample data I used... though it doesn't matter. You can put in your own values and be able to verify what happened easily.
The Heavy Lifting: The Stored Procedure for Cloning Records in COPY_ME
This procedure works with 1 or more records at a time with a special identifier in the COPY_REQUEST table. After the task is done, the procedure cleans up and resets the request value again.
create or replace procedure proc_copy_me_request is
c_request_code CONSTANT char(1):= 'Y';
cursor copy_cursor is
SELECT cme.CUSTOMER_ID, cme.CUSTOMER_NAME, cme.CITY, cme.COUNTRY,
cme.COPY_REQUEST
FROM copy_me cme
WHERE cme.COPY_REQUEST = c_request_code
FOR UPDATE OF cme.COPY_REQUEST;
BEGIN
FOR i in copy_cursor LOOP
INSERT INTO copy_me (customer_name, city, country, copied_from)
VALUES (i.customer_name, i.city, i.country, i.customer_id);
UPDATE copy_me
SET copy_request = null
WHERE CURRENT OF copy_cursor;
END LOOP;
COMMIT;
END proc_copy_me_request;
There is also a column that can be hidden. It tracks where the record was originally copied from.
Note that the cursor is using the FOR UPDATE OF and WHERE CURRENT OF notation. This is important because the procedure is changing the records that are referenced by it.
APEX Page Setup Instructions
Set up a standard FORM type page and choose the TABULAR FORM style. Follow the set up instructions, taking care to map the correct primary key, and also to the PK sequence object created with the table in the previous steps above.
This is what your page set up will look like after these steps are completed:
EDIT The COPY_REQUEST Form Value:
Under the column attributes section, change the Display As option to "simple checkbox"
Under the list of values section, put a single value under the LOV Definition: Y (case sensitive in either way... just be consistent)
EDIT The COPIED_FROM Form Value:
Under the column attributes section, change the Display As option to "Display as Text(Saves State)". This is just to prevent users from stepping on this read-only field. You could also suppress it if it isn't important to know.
CREATE a New Process: Execute Copy Procedure
This is the bottom of the same configuration page, there are very few things to change or add:
Demonstration: Screenshot of COPY_ME Tabular Form Page in Action
The first screenshot below is before the page is tidied up and the checkbox control is put into place.
Plug in some test data and give it at try. The Page Process created in the step above conditionally invokes the stored procedure that processes all copy requests made at the same time when the SUBMIT form button is selected.
COMMENTS: If you spend enough time tinkering around with the built-in wizards in Oracle APEX, there are opportunities to learn new design patterns and process flows compatible within the tool. Adapting your approach can reduce the amount of additional work and frustration.