After Broadcom’s acquisition of VMware in November 2023, significant and abrupt changes were made regarding VMware licensing. These changes, particularly the introduction of excessively high subscription fees and withdrawal of support, pose significant challenges for many businesses. Consequently, numerous companies are exploring alternative solutions to escape VMware’s emerging cost burdens.

However, while fleeing from VMware’s undesirable cost components, companies risk becoming locked into high-cost products that comprise unused components offered by cloud providers.

Hence, in recent months, there has been a considerable interest in virtualization systems built with open-source products. These solutions are gaining traction due to reasons such as the absence of licensing fees, eliminating vendor lock-in, and benefiting from extensive community support.

Understanding the Challenges with VMware

I first encountered VMware in 2002, which was then a Type2 hypervisor. It was seamlessly switching between virtualized instance and your own desktop. Then they introduced their server products and the story began. With their licensed products, they charged a fixed cost per CPU, which was quite affordable for businesses. Support and maintenance services also progressed seamlessly.

However, following Broadcom’s acquisition, VMware suddenly made a series of strategic changes. Among the most significant were the transition from fixed software costs to a subscription model and the announcement that support and subscription renewals for previous perpetual offers would no longer be available.

This essentially meant:

• The transition to a subscription model resulted in a 100-300% price increase for many firms worldwide, exceeding the budgets of most.
• The previous cost calculated per CPU is now calculated per core. While the company claims to have reduced the price per core, it does not significantly affect the overall high price.
• With the subscription system, mandatory purchase of packages containing products that many companies do not need became necessary.
• The company will no longer provide support for previous perpetual sales agreements, and those with such agreements will not be transitioned to the subscription system.

A nice article is written by Nutanix, explaining the 9 predictions of VMware Broadcom acquisition, which can be find here.

These conditions understandably infuriated VMware users worldwide. Early surveys suggest that CIOs and executives are inclined to reject VMware’s imposed new system and all the innovations, support, and services it could offer, focusing on alternative solutions. They are not entirely unjustified. With this decision, thousands of businesses have suffered a severe blow to their short and long-term goals.

Due to the reasons outlined above, there has been a significant increase in interest for the open-source virtualization systems in recent weeks. This article aims to provide information about open-source virtualization alternatives for server virtualization type 1.

Type 1 Virtualization: An Overview

This virtualization method is used by enterprise companies essentially, means running multiple machines within a single physical machine. We call them virtual machines or virtual instances.The physical computer/machine/server on which they reside provides a layer of abstraction for virtual machines, and each virtual machine behaves as if it were a physical machine.

There is also the light version of Type.1 virtualization. This type of virtualization, used mostly on serverless services, is a new technology that has a shorter running time and also shorter boot time. AWS’s serverless solutions, like Fargate, rely on such hypervisors.

Two popular light virtualization options are “Cloud hypervisor” and “Firecracker”. Firecracker is based on KVM and uses very few devices for booting (network, compute, I/O…), only virtualizing necessary devices, results in a boot time of approximately 100-200 ms.

Important reminder: Container Tech is not Virtualization!

On the internet there are many out of context hypervisor comparison charts which includes Container technologies like Docker and Kubernetes as virtualization technologies, that is %100 wrong.”

Containers are not a virtualization technologies. Containers designed to be respawned, virtual instances should run forever. Containers, by design, don’t provide the “isolated abstraction layer”, which is the first requirement on type.1 virtualization.

Exploring Open-Source Virtualization Systems

Before diving in, I’d like to emphasize a general principle which you would already know. If your infrastructure is working smoothly and you have no licensing issues, my suggestion would be “continue operating your systems as they are”. Remember, “the best system is the one you know in details.” In short, keep your systems secure and simple; if it’s an option, continue running VMware.

“Don’t touch, If it works” _ an old programmer

What Are the Alternatives to VMware?

As complexities arise when deploying multiple hypervisors, companies are turning to alternative solutions. For instance, XCP-ng, Proxmox, and Nutanix are mentioned as viable alternatives to VMware.

Hypervisors and Orchestration Tools

Under this section, I will discuss alternatives to VMware suitable for small-scale deployments. By small-scale deployments, I refer to systems where the administrator can easily remember the IP addresses, often used by SMEs. This includes deployments ranging from a 3-node setup to 8-10 compute nodes.

At this scale, it’s essential to run a management tool on your hypervisors. Hypervisor offerings have some excellent solutions that work well in 3-10 node deployments.

However, it’s essential to note a common mistake. If you have more than 8-10 hypervisors in your deployment, say 3 nodes in site 1, 6 nodes in site 2, etc., it doesn’t necessarily mean there are nine hypervisors in total. So, avoid rushing into larger-scale solutions immediately. I highly recommend seeking consultancy in such cases as each deployment has its own dependencies and constraints. Your situation may be more complex than you think.

Alternative Technologies

Hyper-V: Besides Licensing fees and vendor lock-in, Hyper-v is also a viable alternative. Their technology comes from the “Connectix, VirtualPC” which was acquired by Microsoft. vhd disk technology was one of the key VirtualPC technologies which shaped the virtualization market.

KVM Hypervisor: Setting up a KVM distribution is the simplest and least complex type1 hypervisor deployment. However, it requires Linux OS stack knowledge to get everything set up. Creating a flexible environment demands extensive knowledge and experience on Linux. Most Cloud and Hypervisor solutions on the market use KVM as their default hypervisor.

XCP-ng: This project was initiated just after Citrix Xenserver’s limitations of the community version, led by the company Vates, the same group behind Xen-Orchestra. XCP-ng didn’t gain significant attention, but the project persisted, creating a niche community around its solutions. Since some cloud platforms still use XEN, I personally recommend it. Despite its promising and stable solutions, it now also offers an HCI integrated alternative(Linbit,DRBD). Overall, it’s a noteworthy alternative. We used Xenserver and XCP-ng for the last 15 years on many of our production environments (SAP, Oracle DB, HA clusters, MSSQL servers..)

in 15 years, hypervisor related downtime = 1, solved less than 30 minute

However, XCP-ng has some weaknesses to consider:

• CentOS 7: No longer being updated
• Xen: Facing issues with patches
• Network stack: Quite limited, difficult to implement Linux features.
• Integrated Storage: Beta-level solution

Proxmox: Proxmox is the most promising alternative for small-scale deployments. I recommend it for deployments with up to 8-10 nodes, maybe even more. It offers integrated Ceph and backup solutions out-of-the-box, requiring no license to begin. Proxmox includes all features by default, allowing users to choose between LXC (container) and KVM (full virtualized) instances. If you have fewer than 8-10 node hypervisors, this platform is ideal for you. We’ve used Proxmox for around 10 years across various production environments(SAP, Oracle DB, Windows..). In years we encountered very few issues, especially on clustering services. Having in-house Linux expertise enabled us to solve all our problems. I highly recommend it as a Type 1 hypervisor and orchestration platform.

in 10 years, hypervisor related downtime = 1, solved in 60 minute

Proxmox does have some weak points to consider:

• Cluster services (sqlite & shared quorum)
• Network stack: Linux limitations, they also have a new SDN stack
• Can be buggy on large deployments

Still Beta & Obsolete Projects

1. Harvester (still Beta): Harvester uses KVM as the hypervisor, precisely Kubevirt. Instances are created inside containers, which increases complexity. However, Harvester adopts an HCI approach, offering all necessary components like storage and network. On our testing environments Harvester was consuming a lot of CPU resources without any instances running on the system. Harvester is a promising solution but is still in the Beta phase. Can’t be used on production environments.
2. Kubevirt (no applicable platform): Using Kubevirt alone as a hypervisor replacement is challenging. It’s suitable for those with expertise in Openshift (includes Kubevirt) or Kubernetes looking to use a virtualization stack. However, be prepared to encounter many untested use cases.
3. RHEV (discontinued): Initially by RedHat and later by IBM, this project was migrated to Openshift. While the EOL is announced for 2026, updates will end in 09/2024.
4. Ovirt (discontinued): Ovirt, an open-source RHEV, is still developed but has lost interest. It suffered from poor design decisions (Gluster, NFS, Image mgmt, Network).

If you have multiple compute nodes (more than 10), along with storage and network infrastructure to manage, you should explore Edge Cloud Alternatives. There are four promising solutions in the market, so let’s delve into these options.

Edge Cloud

Edge cloud offers numerous benefits, low latency instances being the most significant. If you need to use GPUs on your operational site and your data’s gravity is at your edge location, considering Managed Edge Cloud solutions is essential.

Opennebula: Small deployment works out of the box, but we never deployed the solution on large scale. The community is very limited, you can’t find many people using this solution. Large deployments would need real support, if they reach to us and share their partner information we would be happy to share them.

Cloudstack: Despite the promises they made on 2010s, Cloudstack didn’t deliver as expected. They aimed the same market of Openstack with a very similar product. On those old days people find it easy to create clouds. Industry (HW vendors) and some open source players did not support them. You go to the crowd, because you know community is strong where the majority is. The idea to support multiple hypervisors and being HW vendor agnostic is not a smart idea. Their default documentation is outdated, and while there are deployments performing well, community support is lacking.

Virtuozzo: A vendor-specific Openstack solution, I have experience with their Onapp product (acquired by Virtuozzo). They expect you to have a storage backend, they also have an HCI deployment out of the box, but we did not test solution. Would be nice if they can share more details about their blueprints. They have a real user base and the main solution provider is the company itself, you can just ask them for the whole managed cloud edge operation. They claim that they offer HCI out of the box but their documentation doesn’t mention which solutions they use. Unfortunately they only have marketing materials for end users.

Platform9: They are in the Private/Edge Cloud market for a long time, they have an extensive Openstack experience. During the last 10 years, I have seen many of their Openstack contributions. Platform9 is using Kubevirt as their hypervisor stack, I did not test any of their solution, but they seem to be promising as a solution provider. Like Virtuozzo there is no hands on deployment possible.

Openstack: Openstack stands out as the only real edge cloud solution in the market. With a robust community, it provides fundamental infrastructure building blocks deployable anywhere, including at the edge of the network. Openstack’s flexibility and modularity allow efficient running of minimal services required at the edge while supporting bare metal, container technologies, and virtual machines robustly. Generally, six types of providers and distributions of Openstack can be distinguished:

• Vendor Specific (Huawei, Fujitsu, etc.): Some major tech companies package and offer Openstack with their own hardware and software solutions.
• RedHat: Known for its contributions to the Openstack project and extensive user base, RedHat provides comprehensive documentation for Openstack and offers support for its licensed distributions.
• Canonical: With its “Sunbeam” solution, Canonical integrates MaaS (Metal as a Service) and Juju, among other tools. It also offers support packages with paid Pro versions. Canonical also has a MicroCloud solution, to be honest we did not use that as a solution and we don’t know the details. LXD containers are mostly jail root containers, but apparently MicroCloud is creating qemu instances.
• OpenStack – OpenSource: As an open-source cloud computing platform, OpenStack has various distributions, typically supported by communities or companies.
  • Kubernetes deployment (starlingX, Helm) – Labeled as somewhat obsolete, StarlingX was introduced as a project integrating Kubernetes with OpenStack. This indicates that StarlingX may no longer be the most current or widely used solution. There is also openstack community supported helm deployment.
  • Ansible deployment (kolla-ansible): Kolla-ansible is a tool for deploying OpenStack using Ansible. It automates and simplifies OpenStack deployments.

VMware to OpenStack Migration

Planning is crucial and essential for a successful migration from VMware to OpenStack. There are three main components that need careful planning:

Compute

• VMware only
• KVM + VMware

Network

• Leaf Spine architecture
• L2 network
  • Storage Network
  • Data Network
  • Backup Network

Storage

• Storage backend (HP, Netapp, EMC..)
• Hyperconverged
  • Vxrail
  • CEPH
• CEPH

Both the source (existing VMware environment) and target architectures (OpenStack-based infrastructure) need to be deployed simultaneously. The components freed up from the source architecture will be used to build the target architecture. Depending on the new network design, both architectures can coexist during the migration process.

The project items for the migration may include:

• Selecting a suitable solution partner.
• Developing a comprehensive project plan.
• Planning and allocating necessary resources.
• Network architecture re-design and approval from the peers
• Obtaining management approval.
• Conducting initial tests.
• Executing the migration in phases, starting with Group 1 instances (20-100, for example).

As a provider of Managed OpenStack Services, Kubedo offers expertise to address your infrastructure migration needs. We can assist you in determining the best approach for your deployment. Starting from the Turnkey Openstack deployment to Vmare to Openstack deployment

For a detailed assessment of your infrastructure migration decision, please check one of our meeting times. Our team will provide our expert opinion to guide your cloud migration journey.

You can always contact us via sales@kubedo.com