Or are we just considering creating a platform of platforms, and do we need to?
Hi, I'm Paul and I write blogs to help process the thoughts in my head... :-)
AKA, the musings (post series link) of a slightly grumpy, battle hardened data engineer, technology strategist and enterprise architect.
Context
Sky computing is a concept that I came across about 9 months ago now. At the time Cloud Formations was/is still consuming all my time so I didn’t have enough bandwidth to really think about it. Thankfully, it has made it to the top of my blogging list and reinforced by conversations I recently had with a financial services customer (amongst other emerging industries) that has multi-cloud regulatory challenges. That said, for additional context, I have the following open questions on my mind:
Can sky computing enable multi-cloud integration and interoperability?
Is sky computing a true paradigm shift for public cloud services?
Could we create sky platform as an abstraction over existing public clouds?
Is it just an abstraction?
An abstraction too far?
A abstraction that is needed?
Does multi cloud (today) really just mean redundant storage delivered by other vendors?
Can we deliver a platform or platforms with the existing technologies available?
Is this playing to the principals of a data mesh and decentralisation?
What & Why
If you haven’t encountered the term before, sky computing refers to the ability to create a virtual layer or a platform of platforms by integrating and orchestrating multiple cloud services from different providers. Sky computing in addition to industry regulations aims to overcome the limitations and challenges of using a single cloud provider. Sometimes resulting in vendor lock-in, performance issues, data sovereignty, and security risks. By leveraging sky computing concepts, the intent is that users can access the best features and capabilities of each cloud service, while avoiding their drawbacks and costs.
Another way we could describe sky computing is to say that it is not a product or service that is offered by a single public cloud vendor. Rather, it is a capability that is enabled by the integration and orchestration of multiple cloud services from different providers and vendors. For example, a user can combine the storage and compute services from Amazon Web Services, the database and analytics services from Google Cloud Platform, and the machine learning and artificial intelligence services from Microsoft Azure, to create a sky computing platform that can handle a variety of tasks and applications.
To caveat, I fully accept that this feels like a theoretical set of ideals given the resources we have available today. But, as open-standards and interoperability become common place. My hope is that these ideals can/will become reality.
How
So, given the high expectations from above. How could we achieve this today? For this level of integration and interoperability, does the user need to use additional software tools and frameworks that can facilitate the communication and coordination across the different cloud services?
Databricks
One possible answer today I think could be Databricks. Specifically with a data and AI bias to the possible use cases. Databricks which is a unified data analytics platform that can run on multiple cloud platforms and provide a consistent and seamless experience for the user. Then we could have a Databricks workspace that combines with Unity Catalogue abstractions and integrate compute/storage from several clouds. That said, it feels like Databricks would still need to be hosted on one of the cloud provider nodes as the primary instance to the overall sky instance. Unless Databricks are going to create their own cloud instance. I would also be happy to argue that this is really just simple data virtualization and not a full enough answer to be considered a sky platform. But hey, this is the reason for blogging to think about it.
Azure Arc
Another possible technology option could be Azure Arc services, thinking about SQL Server databases and how Microsoft allow such an abstraction over your resources from a single management plane. Including, hybrid Azure Stack instances and private hardware. Combining the management capabilities of Azure Arc with the open-standards of Databricks would make for a powerful data platform delivery combination, maybe!
Summary
Is a platform of platforms a better way to describe a sky platform?
A platform of platforms could be used to describe or infer a sky platform as it captures the idea of creating a higher-level abstraction over our existing cloud resources.
A platform of platforms can offer more functionality, flexibility, scalability, and reliability than a single platform, as it can leverage the strengths and mitigate the weaknesses of each individual platform.
A platform of platforms can also provide more choice and control for the user, as it can enable the user to select and switch among the best options and alternatives for each component and service.
Therefore, a platform of platforms is a useful and accurate way to describe a sky platform, as it reflects the essence and the value proposition of sky computing.
That said, I would like to circle back to the original point of this blog given a fuller (not complete) set of thinking above. Sky computing – is it a thing? What do you think?
Many thanks for reading.
コメント