Agreed this was an interesting slide. Crazy to think how some people are talented enough to greatly optimize a program in such a short span of time. Also it's interesting to see that even for experts optimizing a program isn't a linear process. I know Kayvon used to say that the best way to optimize a program is do the simplest things first and measure performance to see if that's enough and we clearly see that here as both Dillon and Andrew do some things that actually cause dips in performance while trying to search for an optimal solution.
I think what's interesting about the autoscheduler is that you can think of it as part of the compiler, and so it's as if we've gone ahead and made more sophisticated compilers / the compilers are doing more and more of the work
This slide is really cool, but it somewhat made me feel dismayed. If an autoscheduler can do it so well, why try to optimize it by hand? When would the autoscheduler struggle in comparison to a human (i.e. why is Max filter an outlier here)?
I'm really surprised by the autoscheduler's performance primarily because I imagined that it used heuristics about the DAG specified by the programmer, the cache size of the machine, and other known characteristics of the computation to determine the best schedule. If this information is also known to the programmer, why does it take the programmer so much longer to even approach the autoscheduler's performance? Is the autoscheduler just able to test / iterate more quickly than a person can?
this discussion seemed a bit self-contradictory -- if halide is made to quickly try out different parallelization techniques, why are only a few people allowed to do it at Google? isn't the point of it to make parallelization more accessible? i bet more than a few google engineers know have taken parallel computing haha
Does this auto-scheduler get used in practice? I presume that for most programs Google ships Dillon and/or Andrew would be able to spend more than an hour trying to optimize them, and thus they would eventually beat the auto-scheduler. But maybe also these tasks are artificially easy so that an hour was a reasonable amount of time.
What might make the most sense would be to have the auto-scheduler run as a baseline and then let programmers like Dillon and Andrew spend their time further optimizing from there.
@rthomp If I recall correctly, it was mentioned that the auto-scheduler has improved significantly since this study was done. Combined with much more complicated filters, I don't think it is feasible to have programmers to iterate on the auto-scheduler's schedule (people who could reasonably do this are in very short supply, and their time is very expensive for what would likely be a very small performance improvement)
I don't find this discouraging but rather quite exciting about the future of programming. It really demonstrates the power and effectiveness of Domain Specific Languages.
This is real cool. I wonder whether machine learning can efficiently prune the search space for bad schedules.
Please log in to leave a comment.
I thought this was a very cool experiment. Kayvon and one of his students gave two of the world's most expert Halide programmers a never-before-seen program, and gave them two hours to optimize its performance. As the slide shows, the Auto scheduler was able to outperform them in two of the three experiments. I'm still impressed with how much performance they were able to eke out in less than two hours!