May 28, 2019 David Hallinan Connectors, Engineering, Storage API The Pitfalls of API Integration Services: SharePoint Edition “If you light a lamp for somebody, it will also brighten your path.”Buddha Here at Kloudless, tens of thousands of hours have been spent learning the intricacies of the APIs we provide connections with. Our developers have toiled long days and nights in the pursuit of a deep understanding of the services we allow applications to integrate with, not only in the aim of bolstering our own product but also for the self-betterment that comes with the true comprehension of an outside service. They say that mastery of a subject or profession comes with 10,000 hours of practice. If that much is true, we like to think we’ve mastered a couple of APIs at this point. In the vein of the quote that preceded this piece, we’d like to light the metaphorical lamp of API integrations for others that come after us. In this article, one in a series of guides aimed to help you navigate the pitfalls of specific APIs, we will attempt to walk you through the trials and tribulations we went through when integrating with SharePoint Online, one of our more popular API offerings, in the hope that you can be prepared for the development speedbumps you might encounter while incorporating SharePoint Online into your application. Sharing Is Caring To A Point SharePoint Online is a web-based collaborative platform created by the Microsoft Corporation to interface with Microsoft Office, the industry-leading text and document editor. SharePoint was released in 2001, and primarily is used for document management and storage, although the platform has come a long way in its 18 years since it’s release and provides a wide scope of unique functionality to the massive amount of companies that utilize it. As of today, SharePoint Online boasts more than 190 million users across over 200,000 customer organizations. The Problem When integrating with an API as large and popular as SharePoint Online, our engineers generally will assume that documentation will be thorough and easily accessible. However, in this case, one of our biggest hurdles right off the bat was missing, incomplete, and unclear documentation. Most users access SharePoint Online directly through their Microsoft Office interface, so the APIs documentation was probably a bit of an afterthought for the company. If most of their integrations are done internally, then Microsoft probably sees diminishing returns from exhausting major work hours on a project like thoroughly documenting the SharePoint Online API. In order to achieve full CRUD support for file and folder permissions, our engineers needed to study the SharePoint Online REST API v2.0 documentation. Another major issue that they encountered were the undocumented differences between the REST APIs docs and the Microsoft Graph API, another Microsoft API that our integrations needed to access to implement all of the functionality we needed from SharePoint. On the surface, they look very similar and are marked the same in their documentation, despite crucial differences in accessing the necessary functionality provided by the 2 separate APIs. This meant diving into Microsoft’s SharePoint Online SDK documentation and testing different requests, as they are not documented in the SharePoint Online REST API specific documentation. The Solution One of our engineers in Taipei, William Ott, was tasked with building out the SharePoint connectors in our Unified Cloud Storage API. Initially, Ott ran into an issue with the differences in the data exposed by the two APIs. Two examples of this were both the Graph and SharePoint REST API returning different content for the same top-level folder, and the listing of groups differing between the separate APIs despite them both being accessed by the same account or group. Basically, there isn’t a 1:1 mapping from SharePoint Online resources to Graph API equivalents, which was something that Ott would now have to put together himself. In order to counter these unexpected returns of data, William did extensive research and testing on the SharePoint API to document the unexpected behavior in responses. He ended up using multiple Microsoft APIs to make sure that the data being returned was, in fact, correct and could be reliably trusted once received. This meant sending an incredible amount of requests to a myriad of different endpoints in the hope that he would find a clue to how the data was organized and sent from Microsoft’s servers. After rigorous testing, William discerned enough of a pattern between data returns in the different APIs to be able to account for the pertinent information being returned reliably, allowing Kloudless to access all of the functionality necessary from SharePoint Online. At the end of the day, despite being plagued with constant obstacles from the Microsoft APIs, he had built full CRUD support for file and folder permissions into our SharePoint Online connectors which our customers were absolutely thankful to be finally able to take advantage of. In Closing Despite the massive amount of time that went into querying so many endpoints, William was successful in implementing this important functionality into the Kloudless Unified Cloud Storage API. As a result, he gained a deep and in-depth knowledge of both the SharePoint REST API and the Graph API which provides Kloudless with a helpful roadmap for going forward when building out additional functionality using these APIs. William was most proud of adding support to connect admin accounts without having to store the user’s credentials, an important feature in a security-minded online world. Often, updating functionality in API integrations is not exactly what you would call a “glamorous process.” There are a profound amount of things that can break when an API changes anything about the format of their returns and dealing with the ensuing chaos of broken functionality can be enough to make some developers throw in the towel. As a result, some developers and product managers choose to go with a unified API, such as Kloudless, for their integrations to save themselves the time and stress that goes along with updating functionality after an API update or versioning change. At the end of the day, however, Ott rose to the occasion and built an integral part of our SharePoint Online functionality in the Kloudless Unified Cloud Storage API. We hope that when deciding to incorporate SharePoint Online functionality into your application, you’ll go in prepared for what to expect thanks to William’s experience. We would love to hear from you about any pitfalls you’ve had while integrating with any of the APIs covered in this series. Please write to us at firstname.lastname@example.org and let us know any and all of the problems you may have encountered while wading through the swamp of API integrations.