Loopback Cloudant adapter creates index on all_fields by default - loopbackjs

Description of issue
Sometime back it was noticed that loopback by default created indexes on all_feilds(on any db in cloudant) if you use loopback-cloudant-adapter. However Cloudant support strongly discourages that practice. We are in fact having a lot of performance issues with Cloudant lately and this was one of the recommendation by Cloudant support. So we were trying to figure out a way to make loopback-cloudant-connector not create index on all_feilds and only create it on the ones that we actually need. In words of one of our developers :
We are trying to modify the existing all-field index to specific properties in cloudant models
To accomplish this we tried modifying the existing design documents in cloudant and added specific index fields and tried to invoke the loopback services. The basic crud opertions failed with error Error: mango_idx :: {no_usable_index,missing_sort_index} and also upon app restart the cloudant design document was overwritten to index on all fields.
We want support on modifying existing design documents to index on specific properties and also disable the default loopback functionality to create cloudant indexes on all_feilds_
Additional information (Node.js version, LoopBack version, etc)
Loopback version : 3.6.0
Node js Version : 6.10
I have also opened a issue with loopback on their public github but wanted to check on stackoverflow too to see if someone has already encountered a similar issue. Given below is a link of the gitbub issue that was opened for this :
https://github.com/strongloop/loopback-connector-cloudant/issues/162

Related

Dynamic database connection in Symfony 4

I am setting up a multi tenant Symfony 4 application where each tenant has it's own database.
I've set up two database connections in the doctrine.yaml config. One of the connections is static based on an env variable. The other one should have a dynamic URL based on a credential provider service.
doctrine:
dbal:
connections:
default:
url: "#=service('provider.db.credentials').getUrl()"
The above expression "#=service('provider.db.credentials').getUrl()" is not being parsed though.
When injecting "#=service('provider.db.credentials').getUrl()" as argument into another service the result of getUrl() on the provider.db.credentials service is injected. But when using it in the connection configuration for doctrine the expression is not being parsed.
Does anyone have an idea how to solve this?
You're trying to rely on ability of Symfony services definition to use expressions for defining certain aspects of services. However you need to remember that this functionality is part of Dependency Injection component which is able (but not limited to) to use configuration files for services. To be more precise - this functionality is provided by configuration loaders, you can take a look here for example of how it is handled by Yaml configuration loader.
On the other hand configuration for Doctrine bundle, you're trying to use is provided by Config component. A fact that Dependency Injection component uses same file formats as Config component do may cause an impression that these cases are handled in the same way, but actually they're completely different.
To sum it up: expression inside Doctrine configuration does not work as you expecting because Doctrine bundle configuration processor doesn't expect to get an Expression Language expression and doesn't have support for handling them.
While explanations given above are, hopefully, answers your question - you're probably expecting to get some information about how to actually solve your problem.
There is at least 2 possible ways to do it, but choosing correct way may require some additional information which is out of scope of this question.
In a case if you know which connection to choose at a time of container building (your code assumes that it is a case, but you may not be aware about it) - then you should use compiler pass mechanism yo update Doctrine DBAL services definitions (which may be quite tricky). Reason for this non-trivial process is that configurations are loaded at the early stages of container building process and provides no extension points. You can take a look into sources if necessary. Anyway, while possible, I would not recommend you to go in this way and most likely you will not need it because (I suppose) you need to select connection in runtime rather then in container building time.
Probably more correct approach is to create own wrapper of DBAL Connection class that will maintain list of actual connections and will provide required connection depending on your application's logic. You can refer to implementation details of DBAL sharding feature as example. Wrapper class can be defined directly through Doctrine bundle configuration by using wrapper_class key for dbal configuration

How to publish all versions of an item to Web DB (Sitecore 8.0+)

The end result:
After publishing an item, all versions will be visible in the WEB DB, instead of the default behaviour of only having the latest item version.
I'm using Sitecore 8.2.170407
After reading this article, I tried to do several things:
Remove both RemoveOtherVersions and PublishTestingVersions processors by doing a <patch:delete />. That did not work
Add my own patch file, which inherits Sitecore.Publishing.Pipelines.PublishVersion.Processors.RemoveOtherVersions and hides/overrides the base Process method. The new method will do nothing.
Image of patch class and Image of patch config
That didn't work either.
Tried removing the whole <publishVersion> pipeline, but that gave an exception when trying to publish an item.
I've not been able to find a solutions anywhere else. I know a similar question has been asked here, but that was in Sitecore 6
Have any of you been in the same situation before? How did you solve the issue?
If you need more information, please let me know.
Thanks in advance
You can't.
The linked answer you provide is still valid. Sitecore "web" database (any database which is a publishing target) stores one and only one version. To modify this, you're going to need to reinvent your own publishing process.
Based on your clarification, it seems what you want to use is standard Sitecore versioning with specified publish start dates to put up a particular version.
All of the versions would live in your master database, and you would regularly run publishes through the publishing agent or some other means in order to send the appropriate version to the web database on the correct date.
Your authors can preview what the page would look like on a particular date using the date picker in the preview mode.

Zend Framework 2 and Doctrine change database per module

I have an application which use Zend Framework and Doctrine.
I want to change for a module the database from the default settings.
I have created an alternative connection for doctrine.
When creating/updating the tables using,
./vendor/bin/doctrine-module orm:schema-tool:update --force
the tables are created in the first configuration of database.
Basically what I want to update the second configured database tables.
Can someone help me with an working example ?
Thanks,
Bogdan
To my knowledge, the schema-tool binary only works with the orm_default database.
Now, there's certainly nothing stopping you from having modules that add additional named connections. See this documentation for doing that:
https://github.com/doctrine/DoctrineORMModule/blob/master/docs/configuration.md#how-to-use-two-connections
But, the tooling around managing those additional databases might be a little "roll your own". The good news is all the pieces are there (Doctrine's underlying SchemaTool classes), you would just need to wire them up and build a cli command that acts on multiple schemas.
All that being said, if you find yourself using multiple unique schemas in the same database engine (unique being the key word to account for things like doctrine sharding), I worry your application design might be potentially troublesome. It could be possible that your multiple storage domains should actually live as separate applications.

Sitecore 8.1 xDB data capture requirements

We have just upgraded our 7.2 platform to 8.1. We have enabled xDB as well.
I've following questions:
Do we need to write any custom code (JS or C#code) to capture analytics data on to xDB?
What sort of data is captured by default and what sort of data requires custom code?
Thanks.
1) No custom code is required by default. You just need to make sure that configuration files are properly setup. Sitecore Analytics and xDB features are enabled when you install Sitecore. In Sitecore 8.0 you only need to have "Analytics.Enabled" set on "true" in Sitecore.Analytics.config but in Sitecore 8.1 because they have introduced the notion of separation of xDB and core sitecore functionality you also need to have the extra license for xDB and having "Xdb.enabled" in Sitecore.Xdb.config as well. Also make sure that you have an installed and running MongoDB on your machine since xDB is actually consisted of MongoDB and SQL server (both)
Also have a look on following links about CMS-only mode in Sitecore 8.1:
CMS-only mode: an overview
Sitecore 8.1: what does new CMS-only mode mean
2) Sitecore xDB collects visitors' information in "Contacts" collection on MongoDB and the actual visits in "Interaction" collection on MongoDB (in JSON format) and then it processes raw data to generate statistics and store them into SQL server (separate database for analytics). In general, Sitecore shows you various statistics based on "PageViews" and "Engagement Values" side by side on dozens of charts. Checkout "ReportDataView" and "TrafficOverview" views on SQL server (once you setup xDB up and running) to have some ideas about what is it doing.
Anyway, in many cases you may find the ready-to-use charts and graphs are not enough so you can also have direct access to raw data in MongoDB or aggregated counterpart in SQL server and you can also write your extra pieces of info on each page so that you can extract them later on Experience Analytics.

How do I clean xDB in Sitecore?

Have recently tried working with xDB in Sitecore 8 and now looking for the way of cleaning out current stats from xDB without re-installing Sitecore. I deleted data files for Mongo (as was suggested) but still see figures in Analytics in Sitecore; also did iisreset but also did not help. What am I doing wrong? (I am new to Sitecore so might be missing something).
Have you tried to clean-up only MongoDB files, without Reporting database?
If yes, I think that is a point of your confusion. The way it works in xDB is that all tracking analytics data is written into Mongo and then by SessionEnd processed and saved into Reporting database, that is SQL database, same way as it was before previously in DMS. In that case you need to clean that database as well.
If you have access to SQL, you may use __DeleteAllReportingData stored procedure as the quickest:
More correct approach that goes well for instances where there is no direct access to DB is using admin tool for that located at /sitecore/admin/RebuildReportingDB.aspx. Also there was a module Analytics Database Manager previously, however I do not know its current state.
Reference: Walkthrough: Rebuilding the reporting database (from official documentation)