Optimizing Distributed Systems With Quantum Computing

Abstract

As systems become more complex and large-scale, there is a need for more effective scheduling algorithms to maximize resource usage and minimize job completion times. The older methods of scheduling like First-Come-First-Serve (FCFS) and Round Robin (RR), although easier to compute, tend to perform poorly in dynamic scenarios because they cannot manage intricate constraints and multi-objective optimization efficiently. As traditional methods approach their limit of performance, quantum computing presents a potentially fruitful avenue for conquering combinatorial challenges based on its ability to examine many states at once. This work proposes a quantum-aided scheduling framework that employs Quadratic Unconstrained Binary Optimization (QUBO) formulations and quantum solvers for the purpose of improving job scheduling in distributed environments. The original contribution is the marrying of quantum optimization (with IBM Qiskit's QAOA and D-Wave's hybrid solvers) with traditional distributed computing frameworks such as Apache Spark. Google Cluster Workload Traces are employed to mimic real-world situations, and jobs are described by CPU, memory requirements, and arrival times. Once the scheduling problem has been formulated into a QUBO model, quantum algorithms produce the optimized job-to-machine assignments. The outcomes are compared against baseline classical schedulers on performance metrics including make span, throughput, and resource utilization. The experiment demonstrates that quantum-assisted scheduling significantly improves makespan and enhances resource utilization, particularly under high workloads. It also provides evidence of the feasibility of integrating quantum solvers with existing big data infrastructures.