actually I am working with PDI 8.2, however I am able to upgrade to 9.0.
The main issue is that a customer wants to pull data from salesforce which works well so far. But he is using the Enterprise Web Services API with version 48.0, latest Pentaho supports v47.0 only.
I strongly assume that reading via v48.0 won't work with PDI so that I have to build a workaround. Could anyone point me to a feasible solution? To be honest, I don't even know whether the Enterprise or the Partner API is relevant for Pentaho. Have got my own SF-Account so that I could try around with the APIs.
Is the "Web Services lookup" the right step for the workaround?
Any answer would be appreciated! Thanks in advance!
Oh man, what a crazy question, all over the place.
I strongly assume that reading via v48.0 won't work
You'd have to try it but it should work. Salesforce has 3 releases a year and that's when they upgrade API versions. We're in Spring'20 now, it's v.48. That doesn't mean anything below is deprecated. You should have no problems calling with any API version >= 20. From what I remember their master service agreement states that API version released will stay up at least 3 years. Well, v.20 is 9 years old and still going strong...
Check for example https://baa.my.salesforce.com/services/data/ (if your client has "My Domain" enabled you can use that too instead of some unknown company), you should see a list similar to this: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_versions.htm (no login required, that'd be a chicken & egg situation. You need to choose API version you want when making the login call).
So... what does your integration do. I assume it reads or writes to SF tables (objects), pretty basic stuff. In that sense the 47 vs 48 won't matter much. You should still see Accounts, Contacts, custom objects... You won't see tables created specifically in v 48. Unless you must see something mentioned in Spring'20 release notes I wouldn't worry too much.
If your client wrote a specific class (service) to present you with data and it's written in v.48 it might not be visible when you login as v.47. But then they can just downgrade the version and all should be well. Such custom services are rarely usable by generic ETL tools anyway so it'd be a concern only if you do custom coding.
whether the Enterprise or the Partner API is relevant for Pentaho
Sounds like your ETL tool uses SOAP API. Salesforce offers 2 versions of the WSDL file with service definitions.
"Partner" is generic, all SF orgs in the world produce same WSDL file. It doesn't contain any concrete info about tables, columns, custom services written on top of vanilla salesforce. But it does contain info how to call login() or run a "describe" that gives you all tables your user can see, what are their names, what are columns, data types... So you learn stuff at runtime. "Partner" is great when you're building a generic reusable app that can connect to any SF or you want to be dynamic (some backup tool that learns columns every day and can handle changes without problems. Or there's a "connection wizard" where you specify which tables, which columns, what mapping... new field comes in - just rerun the wizard).
"Enterprise" will be specific to this particular customer. It contains everything "Partner" has but will also have description of current state of database tables etc. So you don't have to call "describe", you already have everything on the plate. You can use this to "consume" the WSDL file, generate your Java/PHP/C# classes out of it and interact with them in your program like any other normal object instead of crafting XML messages.
The downside is that if new field or new table is added - you won't know if your program doesn't call "describes". You'd need to generate fresh WSDL and consume it again and recompile your program...
Picking right one really depends what you need to do. ETL tools I've met generally are fine with "partner".
Is the "Web Services lookup" the right step
No idea, I've used Informatica, Azure Data Factory, Jitterbit, Talend... but no idea about this Pentaho thing. Just try it. If you pull data straight from SF tables without invoking any custom code (you can think of SF custom services like pulling data from stored procedures) - API version shouldn't matter that much. If you go < 41.0 I believe you won't see Individual object for example but I doubt you need to be on so much cutting edge.
Related
I am trying to wrap my head around Microsoft Cloud for Sustainability. Apparently it's a solution based on Microsoft Dynamics. I need to have more back-end to that solution, because as it is right now I'm either lacking permissions (or extra paid access to Microsoft resources) or missing a chunk of documentation, because I'm unable to:
Change default language across the board - I can switch MS Dynamics to any language I want, but it will work for a shell only. Anything that's CfS specific, is in English. Do I remove the demo data and import my own scopes and data? As only thing available are database and Cube for BI analytics and JSON files describing CfS structure in general (that's in CDM), do I really have to create it from scratch? This brings me to second question:
Access entry-level data that's already in demo version - I need to see what's in the database the CfS is using, or be able to modify it. Is there any way to get to it via Business Central, if at all possible?
Since I will be preparing several presentations for potential customers, I need a way to quickly create a dataset based on initial and very basic information provided by each customer, how can I do that with trial user
I work for a company that's Microsoft Certified Partner, so logically resources for what I need should be available to me, but either links in the documentation are dead (and some are, as they redirect to general info) or require some special access level (or are dead, but error message is really not helpful at all).
Is there somewhere else I can go? The Documentation page offers little towards what I need...
P.S. I think I should tag this question with CfS specific tags, but not enough rep...
I have a Coldfusion application, developed without any framework and almost no architecture.
I'd like slowly migrate some part of the application to some kind of java based web application framework.
The original application must still be the main application all the way until the end of the migration.
The application has user and sessions and a lot of functionalities, not easy to decouple.
I'm looking for different ideas.
For example I could try to develop a REST API backe nd and start to use it from the ColdFusion application until I'll have Coldfusion only as front end. This process must go hand by hand with database refactoring and migrations, ie. new API must use its own database, so I guess that decoupling of database will be necessary (and probably database synchronization issues will arise)
But I would like to switch also to a different front end technology.
The situation would be like: Users login in coldfusion, enter main dashboard in Coldfusion and next "some" functionalities will be handled by another framework. That means a template engine is able to understand the user, his roles, his permissions, reproduce the same graphical layout, but it should serve the content with another technology.
Final result must be that all functionalities are migrated to the new technology.
I mention Java as it is in some way related to ColdFusion, but any web application framework could be used in principle.
I also think that Coldfusion is used in the original application is not relevant. The fact that no framework is used, probably gives me more flexibility.
Any architectural suggestion is welcome.
I recently did this very thing. A spaghetti code app developed by a graphic designer with a CFML book, a MSSQL database so denormalized not even Bizarro could have wrestled it and won.
What we did was we left the old app in place, built a new one and tested it extensively, perfected it, made sure it was capable of evolving (and it has). Then we flipped a DNS switch and sent out an announcement that the new system is the only option going forward.
I then built an archive feature that provided read-only access to the mess that we left behind. Some day, if the app owners want to migrate that data, we can try that (it won't go well), but generally the vast improvements have convinced everyone that the old stuff is only needed in that read-only mode.
Obviously that means reporting only goes back to the first day of the new tool, but it was either that (with a 4 week dev time) or they get it all and it takes 8 months doing what you described.
It's worked very well for us, and it's probably been the best-received work I've done here.
The title is probably not be explicit enough, so let me try to explain. I'm working on a new project, built on .NET, it consists of WPF clients that use WCF web services to access an Oracle database. The problem is not this basic architecture, but rather how it's supposed to work with what's already in place.
Currently, applications are written using PowerBuilder and connect to the database directly. Additionally, they use Oracle's SELECT .. FOR UPDATE statements extensively to manage concurrency by locking records. Since the new applications must exists side-by-side with the old ones, they are supposed to lock records in a similar manner too, but the new architecture that relies on web services does not make this easy.
For the moment, what we are thinking of doing is build a "data server" that would be called by the web services and would be responsible for accessing the database. The purpose of this server is to maintain the open connections/transaction needed to maintain record locks throughout several web service calls. This is needed because the "select" and the subsequent "update" parts of the operations that require SELECT .. FOR UPDATE are most likely going to happen in at least two separate web service calls ("get record" and "post update".)
I've searched the Internet for documentation regarding this kind of situation, but I can't seem to find much on the subject. Can this—i.e. keep record locks open throughout several web services calls—be done? Properly? Is my approach appropriate? Are there any published "best practices" on the matter?
Update: The original title of the question was How to maintain record locking with a service oriented architecture? I changed it following John's suggestion, hoping it may inspire some answers.
In general the SOA already have XA transaction turned on. So you can benefit from this and use the UPDATE / SELECT / UPDATE strategy. Meaning that say, you do UPDATE/SELECT as a single INVOKE example is:
UPDATE MY_TABLE SET state='Working' ... ;
SELECT * FROM MY_TABLE WHERE state='Working' ...
Then you can process the data with out fear of someone else will claim that data since it already in a different 'state', provided that the service you write only one in existing for each table.
Finally, you can complete it with the UPDATE ... SET ... state='Complete' where state='Working'. By the way, this is the same strategy the DatabaseAdapter use for polling.
I have created a small game in Java and I would like to add the ability for a player to publish his highscores online.
I'm willing to write the server software myself (it's easy these days with Ruby Mongrel, or even C++). I just need to have some sort hosting. One solution that immediately comes to mind is Amazon EC2. But that's kind of expensive for my needs. Since the requirements are very minimal (I don't even need a website, just a web service) I think there may be a cheaper solution out there.
Does anyone know of a free or cheap provider for this kind of thing?
Update
For those interested, this is solution I came up with:
a SliceHost
purchased a domain name
C++ HTTP server
built upon the Poco HTTPServer
uses SQLite database via Poco Data
Server implements a REST API supporting
High Score table
/hs content type deduced from accept header
/hs.xml forces xml
/hs.txt forces plain text
/hs/add html form, does a POST using XMLHttpRequest
/hof Hall of Fame, content type deduced from accept header
/hof.txt forces plain text
/hof.xml forces xml
game: my own Tetris clone written in Clojure
Something like Slicehost or any other small-scale VPS provider could probably work. You might even be able to write it as a small app and publish it on Google App Engine, which is free up to a certain point.
google app engine comes to mind: http://code.google.com/appengine/
This'll be my first question on this platform. I've done lots of development using Flex, WebORB and ASP.NET. We have solved Concurrency problems with messaging (Pessimistic Concurrency Control). This works pretty good but it also makes the whole application dependent of the messaging. No messaging, no concurrency control.
I know that ASP.NET has version control in DataSets, but how would you go and use that if you are working on a RIA. It seems hard to go and store each dataset in the session of the client... So, if the Client would like need all products, I would need to store the dataset in the session of the client. When the client would change something to a product and save the product, I could then update the dataset (stored in the session) and try to save it...
Seems a lot of work and a lot of memory that will be used (because those products will be kept in the memory of the client, so the dataset needs to be kept on the server side session).
I think the most easy way would be to provide all DTO's with a version number. If the client would try to save a DTO, I could compare the version number with the one in the database.
Lieven Cardoen
This is something I've done before - as the original data was coming from an SQL Server database we just used a rowversion typed column in each DTO to determine if it had changed while the user was working on it.
At this point you can either barf on the error or try and figure out a way to merge the changes, but at least you can tell that it's changed underneath you :)