The graph above shows that multi-node distributed processing might not be more efficient than the single node processing because distributed computing generally brings additional overhead from communications among nodes. This can be verified from inspecting the single thread single core performance on SSD compared to Spark with 128 nodes (the former takes 300s whereas the latter takes 857s)
The graph above shows that multi-node distributed processing might not be more efficient than the single node processing because distributed computing generally brings additional overhead from communications among nodes. This can be verified from inspecting the single thread single core performance on SSD compared to Spark with 128 nodes (the former takes 300s whereas the latter takes 857s)