Exception: An attempt was made to access a socket in a way forbidden by its access permissions - azure-webjobs

When i am deploying my WebJob to Azure i get the following error message:
Unhandled Exception: Microsoft.WindowsAzure.Storage.StorageException: An attempt was made to access a socket in a way forbidden by its access permissions
The stacktrace doesn't help either see:
Unhandled Exception: Microsoft.WindowsAzure.Storage.StorageException: An attempt was made to access a socket in a way forbidden by its access permissions ---> System.Net.Http.HttpRequestException: An attempt was made to access a socket in a way forbidden by its access permissions ---> System.Net.Sockets.SocketException: An attempt was made to access a socket in a way forbidden by its access permissions
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
at System.Threading.Tasks.ValueTask`1.get_Result()
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteAsyncInternal[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)
--- End of inner exception stack trace ---
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteAsyncInternal[T](RESTCommand`1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)
at Microsoft.Azure.WebJobs.Host.Queues.Listeners.QueueListener.ExecuteAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Extensions.Storage\Queues\Listeners\QueueListener.cs:line 168
at Microsoft.Azure.WebJobs.Host.Timers.TaskSeriesTimer.RunAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Timers\TaskSeriesTimer.cs:line 147
at Microsoft.Azure.WebJobs.Host.Timers.WebJobsExceptionHandler.<>c__DisplayClass3_0.<OnUnhandledExceptionAsync>b__0() in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Timers\WebJobsExceptionHandler.cs:line 54
at System.Threading.Thread.ThreadMain_ThreadStart()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
Anybody have a general idea what could solve this?

I've recently run into the same problem.
Webjobs error out when their function can't correctly load their triggers.
As opposed to functions, Webjobs are really not quite talkative about what exactly went wrong when their bindings fail.
I'd check the following:
Make sure you connection string is defined in your appsettings, Azure Config or Keyvault (or wherever you store your connection strings)
Make sure the connection string is called "AzureWebJobsStorage" as the default Attributes (QueueTrigger) only look for that specific key in the config
Make sure that the connection string is actually pointing to a storage that your webjob is allowed to access (for instance, don't point your dev subscription to your qa environment, that was my mistake)

Related

.Net core app runs with EXE but does not run on IIS

I have a .Net Core 6 app which runs fine locally, it also runs fine when I manually start the exe in the publish folder.
However when I run on IIS, it gives the following error.
The server is an AWS EC2 server, I have all ports open as a test as incoming on the firewall, I have also tried to disable the firewall completely and also allowing the app through the firewall. As far as I'm aware there is no AV on the EC2 instance.
I have added the .net core 6 runtime and I have also installed the .NET Core hosting bundle found here:
https://dotnet.microsoft.com/en-us/download/dotnet/6.0
When I have checked the event viewer logs I get this error message:
Application: WebApi.exe
CoreCLR Version: 6.0.222.6406
.NET Version: 6.0.2
Description: The process was terminated due to an unhandled exception.
Exception Info: System.IO.IOException: Failed to bind to address http://localhost:4000.
System.AggregateException: One or more errors occurred. (An attempt was made to access a socket in a way forbidden by its access permissions.) (An attempt was made to access a socket in a way forbidden by its access permissions.)
System.Net.Sockets.SocketException (10013): An attempt was made to access a socket in a way forbidden by its access permissions.
at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException (SocketError error, String callerName)
at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.Bind(EndPoint localEP)
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass30_0`1.<g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.Kestrel.Core.LocalhostListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
I have read there is a setting on old versions of .NetCore to specify the use of IIS, but I can't see one for .Net Core 6.
I am pulling my hair out trying to figure out what is wrong.
Here is what is shown when I run the exe on it's own, which is correct.
I have finally found the reason.
It was due to my program.cs file having this line at the bottom:
app.Run("http://localhost:4000");
I changed this to the following and changed the port to 5000 on IIS and it all started working
app.Run();

Sitecore Team Development Service (TDS) will not synchronise on new Windows 10 machine with VS2017

I have setup a new development machine on Windows 10 with VS2017 and Sitecore 8.2 rev 170728 and am experiencing issues with TDS.
The TDS projects load, the Sitecore connector installs and I can see the _DEV folder and related files in the website folder but when I try and synchronise or perform the test from the TDS project properties I get a 503 error.
Exception The HTTP service located at http://sitecore/_DEV/TdsService.asmx is unavailable. This could be because the service is too busy or because no endpoint was found listening at the specified address. Please ensure that the address is correct and try accessing the service again later. (ServerTooBusyException):
Server stack trace:
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at HedgehogDevelopment.SitecoreProject.VSIP.SitecoreConnector.TdsServiceSoap.Version(VersionRequest request)
at HedgehogDevelopment.SitecoreProject.VSIP.SitecoreConnector.TdsServiceSoapClient.HedgehogDevelopment.SitecoreProject.VSIP.SitecoreConnector.TdsServiceSoap.Version(VersionRequest request)
at HedgehogDevelopment.SitecoreProject.VSIP.SitecoreConnector.TdsServiceSoapClient.Version()
at HedgehogDevelopment.SitecoreProject.VSIP.Utils.Support.CheckClientVersion(TdsServiceSoapClient client, SitecoreProjectNode project, Boolean retry)
Inner Exception Details:
Exception The remote server returned an error: (503) Server Unavailable. (WebException):
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
Exception An error occured getting the TDS service version. Please review the Sitecore logs and/or windows events on the server to determine the problem. (ApplicationException):
at HedgehogDevelopment.SitecoreProject.VSIP.Utils.Support.GetTdsServiceSoapClient(String sitecoreWebUrl, String sitecoreAccessGuid, SitecoreProjectNode project, Boolean checkVersion)
at HedgehogDevelopment.SitecoreProject.VSIP.Utils.Support.GetTdsServiceSoapClient(SitecoreProjectNode project, Boolean checkVersion)
at HedgehogDevelopment.SitecoreProject.VSIP.ToolWindows.SyncWithSitecoreToolWindow.SyncItemsWithSitecore(SitecoreProjectNode project, SitecoreItemNode syncRoot, Boolean sycnChildren)
at HedgehogDevelopment.SitecoreProject.VSIP.SitecoreProjectPackage.ShowSitecoreSyncWindow(SitecoreProjectNode project, SitecoreItemNode selectedItem, Boolean syncChildren)
Everything is set correctly and the website is definitely up and running.
When I access the TdsService.asmx page directly I get an error trying to verify the version number
System.Web.Services.Protocols.SoapException: Service is Locked ---> HedgehogDevelopment.Padlock.PadlockException: Service is Locked
at HedgehogDevelopment.Padlock.Locking.CheckLock()
at HedgehogDevelopment.SitecoreProject.Service.TdsService.Version()
--- End of inner exception stack trace ---
at HedgehogDevelopment.SitecoreProject.Service.TdsService.Version()
I have tried uninstalling and reinstalling the Sitecore connector, even tried different versions of TDS, different VS projects and even pointing TDS at a clean Sitecore installation.
I am not seeing any information in windows event logs, IIS logs or the Sitecore logs related to this error.
I have granted full control access to the website folder, the folder the code is in and the inetpub folder.

Frequent HttpRequestException when talking to Cosmos DB from Web App

We have a web service (Azure App Service) deployed to Azure that talks to our Azure Cosmos DB via the standard C# SDK for Cosmos DB/Document DB.
Both - app service and Cosmos DB account/collections - are in the same resource group and in the same location in Azure.
For certain bulk operations where the web service performs a burst of requests to Cosmos DB we are frequently getting errors in the web service when talking to the database:
System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: An attempt was made to access a socket in a way forbidden by its access permissions
at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.InternalBind(EndPoint localEP)
at System.Net.Sockets.Socket.BeginConnectEx(EndPoint remoteEP, Boolean flowContext, AsyncCallback callback, Object state)
at System.Net.Sockets.Socket.UnsafeBeginConnect(EndPoint remoteEP, AsyncCallback callback, Object state)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
[...]
at Microsoft.Azure.Documents.Query.ProxyDocumentQueryExecutionContext.<ExecuteNextAsync>d__0.MoveNext()<---
Each of our ApiController instances statically allocates a single repository class, which in turn fetches a IReliableReadWriteDocumentClient instance in its constructor and holds it for its entire lifetime via
IDocumentDbInitializer dbinit = new DocumentDbInitializer();
Client = dbinit.GetClient(endpointUrl, myAuthKey, connectionPolicy);
So in my understanding we should use only 2 Document DB clients for our 2 repositories in the entire web service.
Things we've tried so far:
throttle the requests on the client during the bulk operation to less than 3/s
reduce the clients ConnectionPolicy.MaxConnectionLimit from default (50) to 20
increase the apps ServicePointManager.DefaultConnectionLimit
None of these measures significantly reduced the number of exceptions we're experiencing.
Any suggestions how to avoid this error in the first place?
Additional Cosmos DB SDK functionality to tune/configure/adapt for our use case..?

Word Automation Service BatchGetSyncJobStatus fails when requesting security token

I'm running a SharePoint 2013 on-premise server on which I have deployed a simple WCF service as a farm solution. The service accepts simple Http post requests that contain single MS Word documents as payload and returns these files converted into PDFs.
The service is accessible via Http to anonymous users. The WordAutomationService is running as Administration user account of the SharePoint server.
The service class creates an new instance of the Microsoft.Office.Word.Server.Conversions.SyncConverter and passes the proxy of the SharePoint's running WordAutomationService into the constructor (together with some ConversionJobSettings). Finally it calls the Convert method on the SyncConverter with the input stream (the Word document) and output stream (the web response which will contain the resulting PDF document produced by the WordAutomationService).
When creating the SyncConverter I don't set the UserToken property because the access to the service is by anonymous users. According to the remarks here https://msdn.microsoft.com/en-us/library/microsoft.office.word.server.conversions.syncconverter.usertoken.aspx this seems to be fine:
The default value for this property is a null reference (Nothing in Visual Basic), which is anonymous.
This setup works fine for small Word documents with a couple of pages and returns the expected PDF files. But as soon as the execution time of the WordAutomationService on the SharePoint exceeds a certain time threshold (around 5 seconds) the service fails because it never returns (which leads to a read timeout on the client).
According to the logs it seems the reason for this is that after some time the synchronous conversion job moves the work into a background process:
Sync Stream job conversion takes too long. Don't wait anymore. Check its status later
It then polls the status of this job on a regular basis by calling ConversionServiceApplicationProxy.BatchGetSyncJobStatus. Unfortunately this call fails because internally it tries to create a new channel to talk to this process and for that asks for a security token. The SecurityTokenService however cannot complete the token request and throws an exception:
An unhandled exception has occurred. The security token request cannot be completed. System.InvalidOperationException: The security token request cannot be completed.
at Microsoft.SharePoint.SPSecurityContext.SecurityTokenForServiceContext(Uri contextUri)
at Microsoft.SharePoint.SPChannelFactoryOperations.InternalCreateChannelActingAsLoggedOnUser[TChannel](ChannelFactory`1 factory, EndpointAddress address, Uri via)
at Microsoft.Office.ConversionServices.Service.ConfigChannelFactory`1.CreateChannel(EndpointAddress address)
at Microsoft.Office.ConversionServices.Service.ConversionServiceApplicationProxy.GetChannel(Uri uri)
at Microsoft.Office.ConversionServices.Service.ConversionServiceApplicationProxy.ExecuteOnChannel(Uri endpointAddress, Action`1 action)
at Microsoft.Office.ConversionServices.Service.ConversionServiceApplicationProxy.BatchGetSyncJobStatus(ICollection`1 ucids, Uri endpointAddress)
at Microsoft.Office.ConversionServices.Service.BatchGetStatusPollingThread.Run()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart() StackTrace:
at onetnative.dll: (sig=37460b31-4453-4365-92f5-3a11c267be48|2|onetnative.pdb, offset=28F56) at onetnative.dll: (offset=15735)
I'm at a loss now how to get rid of the token issue so that the system can create the necessary channel to poll the conversion job status. Any help is highly appreciated. Thanks!
(I can't post the full log because it registers as spam)
I’ve found that, if you were to install SharePoint 2013 on a Domain Controller (a topology that Microsoft said is only good for development but not for production), then the default anonymous user in IIS (IUSR) will not work reliably, and any WCF solution which is accessed via an IIS site that has Anonymous Access configured to use the IUSR account will fail when it attempts to access Security Token Service.
In this case the most expedient solution is to reconfigure IIS to use another anonymous identity, namely the identity tied to the Application Pool.
For example if your site is called NameOfSite, you can run this in an elevated PowerShell:
Set-WebConfigurationProperty `
-Filter /system.WebServer/security/authentication/AnonymousAuthentication `
-Name username `
-Value "" `
-location "NameOfSite"
This solves the immediate problem at hand which is that SecurityTokenForServiceContext fails. However, if you’ve installed SharePoint 2013 on Windows 2012 R2 as a Domain Controller, then it is not over: WordServerWorker actually will not start in this configuration.
I can also confirm, however, that if you were to install SharePoint 2013 on a standalone server (with <Setting Id="SERVERROLE" Value="SINGLESERVER"/> role in the unattended config file), then the entire solution works end-to-end, and WordServerWorker will actually start properly.
Previously, the most relevant (and unanswered) question on this must be this MSDN posting, “The security token request cannot be completed”. I would assume that in that case, the service was only in a meta-stable state, and one of the IIS workers would have previously obtained credentials via NTLM during local testing.
Usually when sharepoint service applications interact with each other, these services maintaining current user context trough wcf calls by using service application framework (SAF). Its allows these services to use SPContext.Current, preserve correlation id between call in logs and so on. When this context is lost, services stopping being able to communicate each other. For example this happens if we have a code that starts a new thread but didn't setup user for newly created thread context.
According to your description your service is anonymous and didn't use SAF to maintain user context, but uses some services that requires existence of that context
The possible solution would be is to use SAF(which is tricky configured WCF in a nutshell) instead of plain WCF services with no authentication
Edit:
One more possible solution may be is wrap your code with RunWithElevatedPrivileges to make your services connects sharepoint with application pool identity

System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at <wsdl url>

I have developed a webservice using a 3rd Party API on Visual Studio 2012. I run tested it on my local computer and everything works fine. I can get the response i want from the webservice. But when i host this webservice on my remote server, i get the below Exception.
System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at <wsdlurl> that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it <serviceurl>:8080
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
at System.Net.HttpWebRequest.GetRequestStream()
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
So is it my server or is it the 3rd party server where the webservice is hosted? I did a telnet to the server but could not connect. My question is why am i able to run my webservice on my local computer but cannot run it on the server?