How to make SharePoint Rock

SharePoint has been around for nigh on 14 years and is by most standards a mature product.  It is a huge and very sophisticated platform, it works well and it is very stable when treated properly.  Sadly very few people really understand this and as a result many SharePoint installations “go wobbly”.  SharePoint experts around the world have quite rightly said SharePoint must be “governed”; organisations must have governance in place to translate business needs to productive, scalable and robust SharePoint solutions.  This is true, but the problem with this suggestion is that it scares the living daylights out of most organisations who are fearful of the cost implications or the rise of a dreaded bureaucracy so they either ignore the advice all together or they get caught up in paralysis by analysis so the outcome is one of three options:

1. Do nothing because no one knows what to do, the “What now?” syndrome
2. Have a go at everything and see what happens which ends up with the “SharePoint is rubbish because it does not do what I want it to do and it keeps falling over ” reputation
3. Do nothing because there is no agreement as to what needs to be done, the “SharePoint is too complicated” syndrome

I have seen these scenarios many times and I have found that SharePoint works best when the organisation appoints a SharePoint Evangelist.  A successful evangelist has the following characteristics:

1. They have the breadth of experience and vision to understand the true needs of the business
2. They understand what SharePoint really is capable of doing
3. They understand SharePoint’s Technical and Information architectures
4. They know how to marry these needs with these capabilities
5. Ideally too they are sufficiently politically savvy to know how to take the organisation (senior management to junior staff) with them, probably using a combination of stick, carrot and training.  Carrots are preferable followed by lots of training.
6. Some may suggest too that they should also be able to walk on water, but this is optional 😉

If there is no such candidate in-house then hire one on contract to fulfil the role and task them to help you find and mentor a permanent member of in-house staff.  The energy and knowledgeable enthusiasm from such a person will more likely drive the successful adoption of SharePoint, improve the ROI and give rise to a “SharePoint Rocks” reputation than a governance plan with no conviction, however well thought out it might be.

Managed Metadata Service not working on SharePoint 2013

I have just been experimenting with the migration of a live production SP 2010 Farm to a test production SP 2013 Farm. I am using the term “production” to signify that it is a properly configured multi-tier farm not an Out-of-the-box, everything on one box demo environment. The aim is to identify and tackle real world migration issues. And guess what I found some! The first of which is here. There are more to follow!!

The source farm is a multi-tier SP 2010 enterprise edition Farm that is hosting 6 web applications all of which used Windows AD 2003, NTLM authentication. The instructions from MS make it clear that the old system needs to be upgraded to Claims Based Authentication. The supported options being to upgrade in the 2010 environment first then migrate to the 2013 environment, or migrate to the 2013 environment first then upgrade. After going through the instructions carefully I decided to take the latter option. I migrated and then upgraded, using PowerShell, it seemed to complete the process without any errors.

I decided to enable the Managed Metadata Service on the new farm. I followed the instructions very carefully – thanks to Todd Klindt for his useful advice here – but it only worked partially. The service was started and the necessary permissions were granted. From Central Administration I could see and manage the Metadata Terms Store as required. However, when I opened up the Term Store Management from the Site Settings page of any Site Collection within my newly upgraded web application I got the following message:

I Googled the term and found a very helpful blog –my thanks to Allen Wang – that told me I had probably not given the necessary access permissions to the MMS’s Application Pool Service Account. I checked and confirmed that I had given it the correct permissions. I checked and re-checked many times. I deleted and rebuilt the Service Application many times checking that I had followed the correct steps to the letter. Still to no avail.

The system did not report any errors, no correlation IDs, no clues at all. I checked the ULS logs to see if I could spot any inappropriate activity at the time the system was generating the above screen shot. The best I found was this… I have highlighted what appears to be the most significant elements…..

01/10/2013 12:41:21.88     w3wp.exe (0x36F4)     0x17DC    SharePoint Foundation     Logging Correlation Data     xmnv    Medium     Name=Request (GET:http://Sharepoint:9000/_admin/ManageMetadataProxy.aspx?appid=0b33e7f6-934b-42f0-a48f-628b506a36a4&IsDlg=1)    776af39b-018e-607b-c682-7608b6778667

01/10/2013 12:41:21.88     w3wp.exe (0x36F4)     0x17DC    SharePoint Foundation     Authentication Authorization     agb9s    Medium     Non-OAuth request. IsAuthenticated=True, UserIdentityName=, ClaimsCount=0    776af39b-018e-607b-c682-7608b6778667

01/10/2013 12:41:21.90     w3wp.exe (0x36F4)     0x17DC    SharePoint Foundation     Logging Correlation Data     xmnv    Medium     Site=/    776af39b-018e-607b-c682-7608b6778667

In sheer desperation I deleted my newly created/migrated web application – but kept the Content Databases. I recreated the web application from scratch using the Central Admin menus and thereby creating a fresh Claims Based web application. I reattached the Content Databases and hey presto I could now manage the Metadata Termstore from all the Site Collections.

Lessons learnt? Make sure you have upgraded cleanly and effectively to Claims Based Authentication. Even though you have migrated the Content Databases to the new 2013 schema and upgraded the web application to Claims Based Authentication you may still need to delete and re-create the Web Application and re-attach the Content Databases for the upgrade to be fully effective.

So if you have similar symptoms and some suspicious looking entries like this in your ULS logs then delete the web application but not your content databases. Recreate the web application and re-attach the Content Databases.

I hope this helps you.

SharePoint Content Types made easy….Part 1

The subject of SharePoint Content Types came up in a class recently so I put together a demonstration to highlight what are Content Types, what do they do and how easy it is to set up an application that uses them.

The end user’s requirements were as follows:

• The user wanted to create a Visual Aids document library.
• Whenever a user creates a New Document they are to be presented with a choice of 4 document types – A4 Portrait, A4 Landscape, A3 Protrait, A3 Landscape.
• Each choice of document is to be associated with a different document template (size, content, layout and document type).
• All 4 Content Types require the same metadata columns, namely Business Unit, Issue Date, Extension Number, Renewal Date and Notes.
• The Business Unit column needs to refer to an ever changing list of company locations and departments, the Issue Date can be any valid date, the Extension Number needs to be a maximum of 4 characters, the Renewal Date must be 1 year after the date of the last modification and the Notes column should be free form text.
• They also want to reuse this type of Document Library across many sites.

The solution I proposed was as follows:

Create a series of re-usable Site Columns.  These are columns that are defined in one place (preferably at the top level site in a Site Collection so that they can be used anywhere in the Site Collection).  Associate these Site Columns with a new custom Content Type and then create a new Document Library that uses these Content Types.  To make them re-usable create a Library Template based upon this new Library.

1. First create the List of Business Units in the top level site.  This can be a simple Custom List with just one column, the Title column.  I suggest a List of this type of functionality is called Lookup Business Units.
2. Populate this List with data.  Test data will do for now.
3. Next create the Site Columns.  Start with the Business Unit column, make it a Lookup column, and set it to lookup the Lookup Business Units list and display the Title column.
4. The Issue Date is a simple Date Only column
5. The Extension Number should be a Single Line of Text column and that should be restricted to a maximum of 4 characters.
6. The Renewal Date is a calculated column using date Modified plus 356 days.
7. The Notes column is a simple Rich Text, Multiline Text column
8. Next create a Content Type called Visual Aids Documents based upon the standard Document type and add the Site Columns.  Then create the 4 Content Types we need one for each of the size and orientation combinations.
9. Associate each of the 4 new Content Types with a different document template.
10. Then create a new document library called Visual Aids Master.  Enable use of Content Types.  Add the 4 new Content Types and delete the Document content type.
11. The new Content Types are now ready to be tested.
12. Assuming the testing goes well you are now ready to make a Library Template from the Visual Aids Master so that it can be reused throughout the Site Collection.

You are also now in a position where, if you wish to change the Site Columns, add new ones etc all you need to do is modify the settings for the Content Type and/or the Site Columns and they will automatically be applied to all the Libraries using these Content Types.

SharePoint is… (Part 2)

In Part 1 I suggested that SharePoint is all about Lists and the fact that these Lists are managed entirely via the Browser – I think I should enhance that statement and add three more charactertistics.  SharePoint is all about:

1. Lists – of anything – names, addresses, phone numbers, tasks, projects, events, documents, pictures, videos etc
2. Browser – These lists are managed entirely via the Browser – list creation, adding items, amending items, deleting items, creating filtered and sorted views of these lists – all done via the Browser
3. Libraries – these are just Lists of objects such as Word documents, PowerPoint slides, audio files, video files etc.  The difference between Lists ad Libraries is that Libraries have extra capabilites to handle the creation and display of these object.  Capabilities such as Check In / Check Out file locking to ensure only person updates a file at any one time and special display capabilities such as Picture Libraries that enable the display of images in a slide show.  In all other respects Lists and Libraries are the same offering the metadata tagging facilities to classifiy items.
4. Search –  SharePoint’s search engine can crawl, index and answer queries not only about SharePoint held data but also file shares, websites and databases such as SQL and Oracle.
5. Workflow – out of the box workflows monitor the contents of lists and kick off numerous actions depending upon what happens to the contents of the list concerned – but more importantly users can also write and initiate their own workflows.

When you understand this you begin to understand SharePoint.  Judicious manipulation of these 5 characteristics mean that SharePoint can quite justifiably handle Document Management, Records Management, Project Management, Content Management and host Intranets, Extranets and public websites to name but a few of its capabilities.

In Part 3 I will show you how these 5 characteristics deliver all 6 pillars of Microsoft’s benefits wheel (sites, composites, communities, insights, search, content) and much more besides.  So when someone asks you “Can SharePoint do xxxx?” you can give a considered and knowledgeable answer.

BCS on SharePoint Foundation 2010

One of the features of SharePoint 2010, Business Connectivity Services (BCS) that enables the incorporation SQL data in to a SharePoint site is now possible with the free (Foundation Edition) of SharePoint 2010.  However, if you read the small print one of the essential services required to make this viable in a real world environment, the Secure Store Service, is specifically excluded from the Foundation Edition.  The Secure Store Service overcomes the SQL “double hop” problem and enables SharePoint to interrogate SQL databases residing on external SQL servers (ie SQL databases external to the SharePoint SQL database).

Despite this it came to my notice that some of my clients using the Foundation Edition actually had the Secure Store Service installed and working and they had not forced any illegal uploads!  Further investigation showed that those with the Secured Store Service had all installed Search Server Express 2010.  When I tried to mirror this configuration on my own network the Secure Store Service was certainly installed, but when run it reported “the trial version of this software has expired”.  A quick Google search revealed that Microsoft had released a patch to fix this bug – I installed the patch and voila my Secure Store Service now worked on SharePoint Foundation 2010 and I was able to access remote SQL databases.

So my advice to make sure that you have all the features of Business Connectivity Services (BCS) on SharePoint Foundation 2010 is:

1. Install Search Server Express 2010 on to your SharePoint Foundation 2010 web front end server
2. Install the Microsoft patch to fix the “The trial period for this product has expired” problem with the Secure Store Service –  Microsoft KB2276336

Let me know how you get on….

SharePoint 2010 – installing BCS from scratch

One of the “big” new features of SharePoint 2010 is its ability to give users read/write access to SQL databases, but along with these features come many complexities.  Having spent a head banging 24 hours trying to make the Business Connectivity Service (BCS) feature work in a simulated production environment – complete with all the appropriate security settings – I thought I would share my findings on this blog since I could not find any one place that explains everything you need to do to make this feature operate as advertised!  I hope it helps others avoid my frustrations ; -)

My environment is this:

• SharePoint Server 2010 Farm (RTM version)
• Web front end running on Windows 2008R2 64 bit
• SQL 2008R2 64 bit back end running on Windows 2008R2 64 bit
• All in the same Active Directory domain
• SharePoint 2010 has been installed along with all the services – including the Business Connectivity Service –  and all the basic functions are working exactly as they should

My overall objectives are to:

• Install Adventure Works on my SQL 2008R2 server
• Give my SharePoint 2010 users the ability to read selected Adventure Works tables and to update selected records in those tables

In summary the steps are as follows:

1. Download and install the Adventure Works sample databases from Codeplex – making sure that you download the latest 64 bit versions specifically for SQL 2008R2
2. Assign the necessary access rights to the Adventure Works database (I used AdventureWorksLT2008R2) to your one of your SharePoint 2010 service accounts
3. Initiate the Secure Store Service
4. Register the above SharePoint service account with the Secure Store Service
5. Create a SharePoint Data Connection to the Adventure Works database
6. Define the External Content Type in relation to the required Adventure Works table
7. Save the resulting definition file
8. Test the results!

Without going in to the rights and wrongs of Microsoft’s security models the whole process has been made all the more tortuous courtesy of the hackers who made it more important than ever to configure live production systems with really tight security.  There are numerous write ups on the security options for the Business Connectivity Services by Brett Lonsdale of Lightning Tools and Chakkaradeep Chandran – The details of how to configure the Secure Store Service are well explained in Technet.

Skype Error 1603 – fixed!

Normally I only write about SharePoint on this blog, but my recent experiences trying to setup Skype for a friend have been such that I want to share the problems and the solution with a wider audience.

Problem:

Skype v3.6 is up and running on a laptop – operating system Windows XP Professional.  User is being encouraged by Skype to upgrade to the latest version (4.2).  User downloads the file from Skype and runs it – immediately the system reports – “Cannot remove  older version 3.8”  – where did version 3.8 come from?  After much searching on the Skype site and other web resources I had to:

• Run a special version of the Windows Installer to un-install Skype
• Manually edit the Registry to remove all (several hundred!) references to Skype,
• Remove all files on C drive with Skpye anywhere in the name

Having done all this I retried installing Skype v4.2 – this time I had the same error message plus a Error 1603 message.

Skype’s support system was terrible – all it could do is send a standard auto generated message designed to look as though it was produced by a human, repeating the same mantra as before.  Following that mantra to the letter did not work and they clearly had no other suggestions to make – apart from “try v5.0 (beta)” – that too did not work and generated the same error messages!!

Solution:

I decided to try another route – what happens if I re-install v3.6 and then upgrade to v5.0?  Result – it worked!!!

My guess is that the Windows Installer is still looking at some other invisible file content that relates to the previous version of Skype and as long as that reference exists it will insist on having it un-installed before allowing the next version to be installed.  By re-installing a clean version of the original software it reset all the necessary file and registry entries.  This may or may not work for you but it did work for me.

My thanks also goes to www.oldapps.com – a splendid website that has copies of all the old versions of Skype available for free download.  A great resource….

SharePoint 2010 – ROI justification “BOGOF”

Yesterday a retail client of mine was seeking advice as to how to justify the use of SharePoint 2010 for its Staff Intranet. The Return On investment (ROI) was not looking good.  Acquisition of the hardware, SharePoint CALs for 3,000 users and the branding costs were making SharePoint look very expensive – certainly as far as the Finance Director was concerned.  This is becoming a common challenge.  The solution is in fact very simple and uses a financing technique which is very familiar to Finance Directors and can help them feel more comfortable adopting SharePoint.  It boils down to a widely used retail slogan “BOGOF” (Buy One Get One Free).  Buy SharePoint 2010 for the Staff Intranet and get a Document Management/Executive Dashboard/Collaboration system etc for free.  Assuming of course that you need one or more of these extra capabilities!

The following picture demonstrates the concept: 

A SharePoint Farm (the hardware and software required to deliver a SharePoint system) will usually have the capacity to deliver more than one application concurrently.  This means that a SharePoint Farm installed to deliver a Staff Intranet could also deliver a Document Management System, Executive Dashboard, Project Management, Social Networking System etc without needing to upgrade the hardware or software.

The Finance Director can then review the costs of the infrastructure independently of the “marginal costs” of each application.  For the Staff Intranet the “marginal costs” include items such as the branding of the site, cost of planning the menus, taxonomy, and populating the pages.  When determining the total cost of any application typically the Finance Director will allocate a percentage of the infrastructure costs to each application.  The calculation of an appropriate percentage is a black art and there is no right answer, but at least this way the initial system will not be saddled with the need to justify the full infrastructure costs.  Each application can then be reviewed equitably in terms of its Marginal Costs and its allocation of the Infrastructure Costs (this process is also known as the “overhead allocation”).

Hopefully this will enable my client’s Finance Director to see that SharePoint really can deliver an excellent ROI.

PS

For the technicians in the audience – these Marginal Costs can also be applied to the Intranet Maturity Model.

SharePoint is… (Part 1)

There are lots of posts on the web asking “What is SharePoint?”  Frequently the answers are all too convoluted and complex.  Quite simply the answer is “SharePoint is a web based list management system”.  The lists can be lists of anything, names, addresses, tasks, events,documents, pictures, discussions, software bugs (!), alerts – the list of lists is literally endless.  What makes SharePoint clever (IMHO) is that the creation and management of all these lists can be conducted entirely via a web browser.  Via the browser you can create Lists (aka tables), add Columns (aka fields), add Items (aka rows) and create Views of these Lists displayed, filtered and sorted however you wish all without the need of any assistance from the IT Dept.  To get you up and running quickly you can start with the templates supplied by Microsoft and then modify them to meet your own needs.  Creating a List only takes a few seconds (literally) no waiting in a queue for someone else to do the job for you.  Anyone familiar with setting up a database in Excel will be very comfortable setting up a SharePoint List.

One difference between an Excel database and a SharePoint List is that a SharePoint List can have files such as documents, images etc (aka binary large objects) attached to Items in the List.  In SharePoint speak these Lists tend to be known as Libraries.  Given that it is possible to upload, download, open and save a document directly to a SharePoint site you have in effect a web based file server.  Add to this the ability to tag documents with additional information and you have the makings of a web based document management system.

Over the 10 years since SharePoint was first launched Microsoft have, quite reasonably, added enormously to it.  Lists can be linked to provide Lookup capability, customised workflows can be applied and alerts issued to users when something new is added or a change is made.  It is even possible to make a SQL table in a database such as SAP viewable and editable as SharePoint List.

The list goes on….

When I find the time I will evangelise further with another blog post….

SharePoint 2010 – Remote Blob Storage Impersonation

If you have lots of large files (more than say 50 Gb) to store on your SharePoint site you will probably be considering the use of Remote Blob Storage (RBS).  This enables your server to store large documents of all types on a cheaper file storage device so you do not need to use the more expensive high performance disks demanded by the combination of SharePoint and SQL.  It is easy to wind up the IT techies by asking them why new server storage space is so expensive when you can buy 1TB disks from Maplins for £70!  RBS offers a plausable option to make use of this cheaper disk space.

Courtesy of a great blog by Todd Klindt I installed RBS on my SharePoint Farm and tried it out.  I uploaded a small 64Kb document and that worked perfectly.  I then tried a 3Mb document only to have the server respond with a message saying that the URL for the newly uploaded document was invalid.  Trawling through yards of log files I came across an entry that suggested an invalid impersonation error!  More trawling and Googling suggested that the servers on my SharePoint Farm need to have identical versions of the operating system.  It seems that most users of SharePoint 2010 are installing all the farm functions on to one box and therefore this requirement was automatically supplied.  Most real world production farms have at least 2 servers (one web front end server and one SQL back end server) and no where can I find any reference to the need to have identical versions of the operating system for this RBS feature to work.

In my experiences so far SharePoint 2010, like its predecessors, is a very stable and reliable system, but this experience does enforce the need to test the system in a fully featured and fully configured test environment before going live on the production farm.

For the benefit of geeks and search engines the symptoms are:

• SharePoint Server 2010 running Remote Blob Storage (RBS)
• Uploading documents less than 1Mb works as it should
• Uploading documents greater than 1Mb generates an on screen error message – URL for <shared documents document name> is invalid the document may have been deleted.
• The web server error logs shows

Exception thrown storing stream in new SqlRemoteBlob: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> Microsoft.Data.SqlRemoteBlobs.RemoteBlobStoreException: The blob store <FilestreamProvider_1> threw an exception. The exception thrown is saved in InnerException. —> System.ComponentModel.Win32Exception: Either a required impersonation level was not provided, or the provided impersonation level is invalid     at System.Data.SqlTypes.SqlFileStream.OpenSqlFileStream(String path, Byte[] transactionContext, FileAccess access, FileOptions options, Int64 allocationSize)     at System.Data.SqlTypes.SqlFileStream..ctor(String path, Byte[] transactionContext, FileAccess access, FileOptions options, Int64 allocationSize)     at Microsoft….

The solution – check the version of the operating system on all the servers in the farm – if they are not all identical then upgrade as necessary.