Develop a Provider-Hosted SharePoint App

Using the steps below, I was able to develop a Provider-Hosted SharePoint App for hybrid Site Collection provisioning i.e. in SharePoint Online as well as on premise. The Provider-Hosted SharePoint App will act as a “stub” for a WCF (Web) Service. This service will be called from a Remote Event Receiver that has been added to a list in the App Web. Once triggered, the Provider Hosted App will make an app-only call back to SharePoint using the C# CSOM to create a new Site Collection.

Important

The following is only valid for a single-server development environment i.e. SharePoint and Visual Studio are installed on the same (virtual) machine.

Prerequisites

1. SharePoint 2013 SP1 is installed
2. SharePoint 2013 CU April 2014 is installed (this will update the CSOM so that Site Collections can be created on premise much in the same way this was before already possible for SharePoint Online).

Step 1: The security plumbing (SSL, Certificate, Token Issuer)

A Provider-Hosted App needs a lot of (security) plumbing to ensure that information is exchanged in a secure way and only by authorized users and apps. Since this is a development-environment, SSL is optional. But of course for any production or staging environment you’ll want to add this extra security layer.

Note Also, when you finally publish your app, you have to define your app’s start page and the VS-Wizard requires this address to start with HTTPS. Since you don’t know the URL of your app at development time, you’ll need to create a (self-signed) SSL certificate for the IIS web application that hosts your provider hosted app.

• Since this is a development environment I can create the required X.509 certificate that I need to provide a mechanism for encryption for the Server to Server communication myself. For a production or staging environment you obviously need to buy a certificate from a recognized issuer. To create such a certificate you have basically two options:
  For the default website without any host nameYou can use IIS Manager > Server > Server Certificates > Create Self-Signed Certificate and follow the description here http://msdn.microsoft.com/en-Us/library/office/fp179901(v=office.15).aspx
  For a host named websiteb. download the IIS 6.0 Resource Kit Tools and use SelfSSL to create a Self-Signed Certificate. More infos I found here: http://www.sslshopper.com/article-how-to-create-a-self-signed-certificate-in-iis-7.html
  To create a certifcate for my website with host name “my-sharepoint.is-sp2013.local” with SelfSSL I used:

  C:\>SelfSSL /N:CN=my-sharepoint.is-sp2013.local /V:1000

• Save the certificates locally (e.g. here: “C:\DEV\Certs\HighTrustAppCert.cer” (= public key) and “C:\DEV\Certs\HighTrustAppCert.pfx” (=private key)) and create a new Token Issuer for it and register it it as a trusted Token Issuer with SharePoint’s Secure Token Service using the following script.

  $publicCertPath = "C:\DEV\Certs\HighTrustAppCert.cer"
  $certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($publicCertPath)
  
  New-SPTrustedRootAuthority -Name "HighTrustAppCert" -Certificate $certificate
  
  $realm = Get-SPAuthenticationRealm
  
  $specificIssuerId = "11111111-1111-1111-1111-111111111112"
  $fullIssuerIdentifier = $specificIssuerId + '@' + $realm
  
  New-SPTrustedSecurityTokenIssuer -Name "High Trust App Cert" -Certificate $certificate -RegisteredIssuerName $fullIssuerIdentifier –IsTrustBroker
  iisreset
  
  $serviceConfig = Get-SPSecurityTokenServiceConfig
  $serviceConfig.AllowOAuthOverHttp = $true
  $serviceConfig.Update()
  

  Issuer ID This is a randomly picked GUID. If you pick your own, make sure to only use lower case characters. You can use this issuer (of secure tokens for your provider hosted app) for more than app. So make sure you’ll copy the output of this script and save it for later.

Step 2 – Create a provider hosted app

• Create a SharePoint App in Visual Studio using the appropriate template and select “Provider Hosted”. Note how a second (Web Application) project was created automatically by Visual Studio. This is the project that is the provider hosted application. For now we can leave it as it is. Don’t remove or delete it yet. We’ll cross that bridge when we come to it.
• Open the AppManifest.xml and make the following edits

<?xml version="1.0" encoding="utf-8" ?>
<!--Created:cb85b80c-f585-40ff-8bfc-12ff4d0e34a9-->
<App xmlns="<a href="http://schemas.microsoft.com/sharepoint/2012/app/manifest">http://schemas.microsoft.com/sharepoint/2012/app/manifest</a>"
     Name="WorkspaceManager"
     ProductID="{b2c6212d-4499-40f1-a9d4-4f66ad1dec75}"
     Version="1.0.0.0"
     SharePointMinVersion="15.0.0.0"
>
  <Properties>
    <Title>Workspace Manager</Title>
    <StartPage>~appWebUrl/Pages/index.html?{StandardTokens}</StartPage>
  </Properties>

  <AppPrincipal>
    <RemoteWebApplication ClientId="226ea038-b455-4e18-b269-43ff36a2730e" />
  </AppPrincipal>
  <AppPermissionRequests AllowAppOnlyPolicy="true" >
    <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
  </AppPermissionRequests>
</App>

Pay special attention toRemoteWebApplication Makes the App into a Provider hosted App

ClientId The Client ID is a GUID that’s unique to the App and that is for all installed instances in all SharePoint Farms globally the same. You need to tell your SharePoint environment that you’ll like to install a remote application by registering it. You can do this using the registration tool: /_layouts/15/appregnew.aspx. Don’t be confused by “Generate” buttons. They can generate a GUID for you but normally you’ll want Visual Studios Tool to do so and register your Client ID in SharePoint and not the other way around.

AllowAppOnlyPolicy Setting this to true, enables the App to be called without a user being logged in. For our scenario, where we want to asynchronously call back into SharePoint, this is a requirement.

• I want my app to integrate in the SharePoint user experience. The user should be able to enter all required data into a form for a regular SharePoint list. Luckily it’s still possible to create an App Web in SharePoint with some lists etc. for the provider hosted app. Hence finalize the App by adding Columns, ContentTypes and Lists as required. To provision Site Collections I want the user to provide me with a few details like Title, Tenant Admin Uri, Target (online or on premise), Site owner, Template and URL so I created a ContentType for this purpose and based on this ContentType created a list for it.

Step 3 – Create a new DLL Project (for the WCF service)

• Now I want to implement a List Item Event that is triggered each time when a new entry is added to the list. To accomplish this, I need to add a Remote Event Receiver to the project. Doing so and naming it WorkspaceAdded will automatically add a WCF service to the Web Application project in the solution.
• Actually, I don’t need the Web Application. I only need the WCF service. Hence I remove the Web Application and instead create a new DLL Project for the WCF service and add a couple of files from the Web Application project back to this project:

Web.Config You want to keep this (Must be in the root of the project), because it has all the plumbing you need for the WCF service as well as a couple of application settings needed for the service to take the identity of the registered provider hosted app:

<appSettings>
  <add key="ClientId" value="7ca21fdf-eaaf-452e-b2ba-8d56ef7c6066" />
  <add key="ClientSigningCertificatePath" value="C:\DEV\Certs\HighTrustAppCert.pfx" />
  <add key="ClientSigningCertificatePassword" value="------" />
  <add key="IssuerId" value="11111111-1111-1111-1111-111111111112" />
</appSettings>

TokenHelper.cs The TokenHelper class is a little gift from Microsoft that deals with the complexity of obtaining and exchanging an oauth token.
SharePointContext.cs For this example we don’t need this class but in the near future you’ll probably find good use for this.
WorkspaceAdded.cs The class implementing the WCF service.
WorkspaceAdded.svc The WCF host file.

• To wire up the app project with the DLL project, some “hocus pocus” is required. Visual Studio doesn’t like the idea of a DLL project being the Web Project that is linked with the app. But if no Web Project is linked with the app, Visual Studio will assume a “manifest-only” deployment scenario and publish the app with a wrong profile. Hence I quickly unload the project und edit the .csproj file by adding (or updating) the following snippet:

<ItemGroup>
    <ProjectReference Include="..\WorkspaceManager\WorkspaceManager.csproj">
      <Project>{1F32CD07-35F3-42BF-8A99-B1E231DDF966}</Project>
      <Name>WorkspaceManager</Name>
      <Private>True</Private>
      <RoleType>Web</RoleType>
      <OutputItemType>SharePointWebProjectOutput</OutputItemType>
      <RoleName>WorkspaceManager</RoleName>
      <ReferenceOutputAssembly>False</ReferenceOutputAssembly>
    </ProjectReference>
</ItemGroup>

Update the ProjectReference and Project GUID as needed.

Step 4 – Enable Remote Site Collection creation for SharePoint

• Enable Self-Service Site Collection Creation using the Central Admin
• Install Windows PowerShell for SharePoint Online to install the correct version of the Microsoft.Online.SharePoint.Client.Tenant.dll here http://www.microsoft.com/en-us/download/details.aspx?id=35588
• Enable Remote Site Collection Creation for the target Web Application using PowerShell

# Enable the remote site collection creation for on-prem in web application level
 # If this is not done, unknon object exception is raised by the CSOM code
 #
 $WebApplicationUrl = "http://my-sharepoint.is-sp2013.local"
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null)
 {
     Write-Host "Loading SharePoint Powershell Snapin"
     Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }   
    
 $webapp = Get-SPWebApplication $WebApplicationUrl
 $newProxyLibrary = New-Object "Microsoft.SharePoint.Administration.SPClientCallableProxyLibrary"
 $newProxyLibrary.AssemblyName = "Microsoft.Online.SharePoint.Dedicated.TenantAdmin.ServerStub, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c"
 $newProxyLibrary.SupportAppAuthentication = $true
 $webapp.ClientCallableSettings.ProxyLibraries.Add($newProxyLibrary)
 $webapp.Update()
 
 Write-Host "Successfully added TenantAdmin ServerStub to ClientCallableProxyLibrary."
 
 # Reset the memory of the web application
 
 Write-Host "IISReset..."   
 Restart-Service W3SVC,WAS -force
 Write-Host "IISReset complete."   



3. Set a "fake" Tenant Administration Site e.g. the root address of the target web application using PowerShell as follows

# Set admin site type property to the site collection using PS for any site collection type.
 # This is needed to be set for the site collection which is used as the
 # "Connection point" for the CSOM when site collections are created in on-prem
 #
 $siteColUrl = "http://my-sharepoint.is-sp2013.local"
 
 $snapin = Get-PSSnapin | Where-Object {$_.Name -eq 'Microsoft.SharePoint.Powershell'}
 if ($snapin -eq $null)
 {
     Write-Host "Loading SharePoint Powershell Snapin"
     Add-PSSnapin "Microsoft.SharePoint.Powershell"
 }
 
 $site = get-spsite -Identity $siteColUrl
 $site.AdministrationSiteType = [Microsoft.SharePoint.SPAdministrationSiteType]::TenantAdministration

If this Site Collection is deleted, this script needs to run again.

Step 5 – Implementing the Remote Site Collection Service

• What follows is part of the code of my prototype implementation

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SharePoint.Client;
using Microsoft.SharePoint.Client.EventReceivers;
using log4net;
using System.ServiceModel.Activation;
using System.ServiceModel;
using Microsoft.SharePoint.Client.Services;
using System.Security;
using System.Web.Configuration;
using My.SharePoint.Client.RemoteSiteCollection;

namespace My.SharePoint.WorkspaceManager.Services
{
    [BasicHttpBindingServiceMetadataExchangeEndpointAttribute]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)]
    [ServiceBehavior(IncludeExceptionDetailInFaults = true)]
    public class WorkspaceAdded : IRemoteEventService
    {
        private static ILog log = LogManager.GetLogger(typeof(WorkspaceAdded));

        /// <summary>
        /// Handles events that occur before an action occurs, such as when a user adds or deletes a list item.
        /// </summary>
        /// <param name="properties">Holds information about the remote event.</param>
        /// <returns>Holds information returned from the remote event.</returns>
        public SPRemoteEventResult ProcessEvent(SPRemoteEventProperties rep)
        {
   // Not implemented
        }

        /// <summary>
        /// Handles events that occur after an action occurs, such as after a user adds an item to a list or deletes an item from a list.
        /// </summary>
        /// <param name="properties">Holds information about the remote event.</param>
        public void ProcessOneWayEvent(SPRemoteEventProperties rep)
        {
            try
            {
                var tenantAdminUrl = rep.ItemEventProperties.AfterProperties["TenantAdminUrl"].ToString();
                var targetString = rep.ItemEventProperties.AfterProperties["Target"].ToString();
                var credentials = GetCredentials(properties.TenantAdminUrl, properties.Target);
                var remoteSiteCollectionUrl = rep.ItemEventProperties.AfterProperties["TargetUrl"].ToString();
                var title = rep.ItemEventProperties.AfterProperties["Title"].ToString();
    var template = rep.ItemEventProperties.AfterProperties["Template"].ToString();
    var ownerAccountString = rep.ItemEventProperties.AfterProperties["OwnerAccount"].ToString();
    
    // properties is an object to hold all variables created before - Code not shown

                Manager.Create(properties, properties.Target, credentials);
            }
            catch (Exception e)
            {
                log.Error("Error occured @ProcessOneWayEvent: " + e.Message + "\n" + e.StackTrace);
            }
        }

        private static SharePointOnlineCredentials GetCredentials(String siteUrl, Targets target)
        {
            SharePointOnlineCredentials credentials = null;

            if (target == Targets.Online)
            {
                var userName = WebConfigurationManager.AppSettings.Get("O365UserName") ?? String.Empty;
                var pwd = WebConfigurationManager.AppSettings.Get("O365Password") ?? String.Empty;
                SecureString sPwd = GetSecurePassword(pwd);

                /* End Program if no Credentials */
                if (String.IsNullOrEmpty(userName) || (pwd == null))
                {
                    throw new ArgumentException("You entered wrong credentials for SharePoint Online...");
                }

                credentials = new SharePointOnlineCredentials(userName, sPwd);
            }

            return credentials;
        }

        private static SecureString GetSecurePassword(String pwd)
        {
            SecureString sStrPwd = new SecureString();
            try
            {
                foreach (char chr in pwd.ToCharArray())
                {
                    sStrPwd.AppendChar(chr);
                }
            }
            catch (Exception e)
            {
                sStrPwd = null;
                log.Error(e.Message);
            }

            return sStrPwd;
        }
    }
 
 public static class Manager
    {
        private static ILog log = LogManager.GetLogger(typeof(Manager));

        public static void Create(Properties properties, Targets target, SharePointOnlineCredentials credentials = null)
        {
            switch (target)
            {
                case Targets.Online:
                    if (credentials == null)
                    {
                        throw new ArgumentNullException("SharePoint Online Credentials are missing...");
                    }
                    CreateOnline(properties, credentials);
                    break;
                case Targets.OnPrem:
                    CreateOnPrem(properties);
                    break;
                default:
                    break;
            }
        }

        private static void CreateOnline(Properties properties, SharePointOnlineCredentials credentials)
        {
            ClientContext ctx = new ClientContext(properties.TenantAdminUrl);
            ctx.AuthenticationMode = ClientAuthenticationMode.Default;
            ctx.Credentials = credentials;

            var tenant = new Tenant(ctx);

            var newSite = new SiteCreationProperties()
            {
                Url = properties.RemoteSiteCollectionUrl,
                Owner = properties.OwnerAccountString,
                Template = properties.Template,
                Title = properties.Title,
                StorageMaximumLevel = properties.StorageMaximumLevel,
                StorageWarningLevel = properties.StorageWarningLevel,
                TimeZoneId = properties.TimeZoneId,
                UserCodeMaximumLevel = properties.UserCodeMaximumLevel,
                UserCodeWarningLevel = properties.UserCodeWarningLevel
            };

            var spoOperation = tenant.CreateSite(newSite);

            ctx.Load(spoOperation);
            ctx.ExecuteQuery();

        }

        private static void CreateOnPrem(Properties properties)
        {
            var tenantAdminUri = new Uri(properties.TenantAdminUrl);

            string appOnlyAccessToken = TokenHelper.GetS2SAccessTokenWithWindowsIdentity(tenantAdminUri, null);

            using (var ctx = TokenHelper.GetClientContextWithAccessToken(tenantAdminUri.ToString(), appOnlyAccessToken))
            {
                var tenant = new Tenant(ctx);
                var newSite = new SiteCreationProperties()
                {
                    Url = properties.RemoteSiteCollectionUrl,
                    Owner = properties.OwnerAccountString,
                    Title = properties.Title,
                    Template = properties.Template
                };

                //start the SPO operation to create the site
                var spoOperation = tenant.CreateSite(newSite);

                ctx.Load(spoOperation);
                ctx.ExecuteQuery();
            }
        }
    }

• Now you can deploy the service by adding its DLL to GAC using a post build event as follows:

“C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\gacutil.exe” /i “$(TargetDir)$(TargetFileName)”

• Don’t forget to also add
  

  C:\Program Files\SharePoint Online Management Shell\Microsoft.Online.SharePoint.PowerShell\Microsoft.Online.SharePoint.Client.Tenant.dll

  to the GAC (or at least enable the service to load it from another location).
• Create an SSL endpoint for the provider hosted app. For this I created an IIS Web Application and named it wcf.is-sp2013.local. I created a virtual directory below it and called it WorkspaceAdded. Using IIS Manager I turned this directory into an application. Using SelfSSL I created an SSL certificate for it. I copied the WorkspaceAdded.svc and Web.Config file.
• Last but not least I added a redirect.aspx file in the root of this Web Application. It will be the splash page for the provider hosted app. The challenge here is that the provider doesn’t know the full URL of the app at development time. Only after installation of the app its full URL is known. However, when the user’s launches the app, the start page is called and you can configure it to be called with all tokens (like host web url, app web url etc.). So a little JavaScript can help here. Here’s my redirect.aspx (it needs to be .aspx because SharePoint will call the start page using HTTP POST and not a simple GET):

<!DOCTYPE html>
 
 <head>
     <meta http-equiv="X-UA-Compatible" content="IE=Edge">
     <title>Workspace</title>
 </head>
 
 <html xmlns="<a href="http://www.w3.org/1999/xhtml">http://www.w3.org/1999/xhtml</a>">
 <head runat="server">
     <title>Welcome</title>
 </head>
 <body>
  <p>
   <span id="hostWebUrlPlaceHolder"></span>
  </p>
 
  <script type="text/javascript">
   function getQueryStringParameter(paramToRetrieve) {
    var params = document.URL.split("?")[1].split("&");
    for (var i = 0; i < params.length; i = i + 1) {
     var singleParam = params[i].split("=");
     if (singleParam[0] == paramToRetrieve)
      return singleParam[1];
    }
   }
   
   var hostWebUrl = getQueryStringParameter("SPAppWebUrl");
   console.log(hostWebUrl);
 
   var a = document.createElement('a');
   var linkText = document.createTextNode("Launch App...");
   a.appendChild(linkText);
   a.title = "Launch App...";
   a.href = unescape(hostWebUrl);
   var placeHolder = document.getElementById('hostWebUrlPlaceHolder');
   placeHolder.appendChild(a);
 
  </script>
 </body>
 </html>

This page will simple show a link that when clicked will redirect the user back to the Workspace List in the App Web.

Step 6 – Wiring up the app and the provider hosted app

• Then I added the remote event receiver to the project, Visual Studio added an elements.xml to the project. Open it and edit so that the URL points to the new service

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="<a href="http://schemas.microsoft.com/sharepoint/">http://schemas.microsoft.com/sharepoint/</a>">
  <Receivers ListTemplateId="100">
    <Receiver>
      <Name>WorkspaceAdded</Name>
      <Type>ItemAdded</Type>
      <SequenceNumber>10010</SequenceNumber>
      <Url>https//wcf.is-sp2013.local/WorkspaceAdded/WorkspaceAdded.svc</Url>
    </Receiver>
  </Receivers>
</Elements>

• Left-click the project and choose publish. Now you can set the app identity and store it a profile in the project by entering details like Token Issuer, Client Id etc. you previously generated to create and register the Token Issuer.

Note If Visual Studio cannot find the Web Application project or the Web.Config file in the root of the Web Application project it will show a different dialog (for only publishing the app manifest).

• Continue by clicking “Package the app” and enter the Startup Page (and again the Client ID).

Step 7 – Deploy the app at Tenant Scope

• Because this app requires Tenant Scoped permissions it needs to be deployed at Tenant Scope. This can be achieved by uploading the published .app file in the SharePoint App Catalogue first and then install the app in the App Catalogue. This seems a bit strange at first, but once the app is installed in the App Catalogue, it can be deployed to other Workspaces.

To deploy the app for example to http://my-sharepoint.is-sp2013.local simply click DEPLOYMENT, add the URL to the collection by entering it in the box for “Enter a site collections to deploy to:”, click Add and OK to confirm.

Further notes

1. To get Log4net working in a WCF service I added its configuration to the Web.Config instead of in own file that would be referenced in the assembly file. Apparently log4net is loaded to late in such cases.
2. Provider hosted apps can only be deployed at Tenant Scope.
3. Force IE to use Standards Mode by adding
<meta http-equiv=”X-UA-Compatible” content=”IE=Edge”>