TY - GEN
T1 - LLVM Code Optimisation for Automatic Differentiation
T2 - 6th Workshop on Data Management for End-To-End Machine Learning, DEEM 2022 - In conjunction with the 2022 ACM SIGMOD/PODS Conference
AU - Schüle, Maximilian E.
AU - Springer, Maximilian
AU - Kemper, Alfons
AU - Neumann, Thomas
N1 - Publisher Copyright:
© 2022 ACM.
PY - 2022/6/12
Y1 - 2022/6/12
N2 - Both forward and reverse mode automatic differentiation derive a model function as used for gradient descent automatically. Reverse mode calculates all derivatives in one run, whereas forward mode requires rerunning the algorithm with respect to every variable for which the derivative is needed. To allow for in-database machine learning, we have integrated automatic differentiation as an SQL operator inside the Umbra database system. To benchmark code-generation to GPU, we implement forward as well as reverse mode automatic differentiation. The inspection of the optimised LLVM code shows that nearly the same machine code is executed after the generated LLVM code has been optimised. Thus, both modes yield similar runtimes but different compilation times.
AB - Both forward and reverse mode automatic differentiation derive a model function as used for gradient descent automatically. Reverse mode calculates all derivatives in one run, whereas forward mode requires rerunning the algorithm with respect to every variable for which the derivative is needed. To allow for in-database machine learning, we have integrated automatic differentiation as an SQL operator inside the Umbra database system. To benchmark code-generation to GPU, we implement forward as well as reverse mode automatic differentiation. The inspection of the optimised LLVM code shows that nearly the same machine code is executed after the generated LLVM code has been optimised. Thus, both modes yield similar runtimes but different compilation times.
KW - GPU
KW - automatic differentiation
KW - in-database machine learning
UR - http://www.scopus.com/inward/record.url?scp=85133202786&partnerID=8YFLogxK
U2 - 10.1145/3533028.3533302
DO - 10.1145/3533028.3533302
M3 - Conference contribution
AN - SCOPUS:85133202786
T3 - Proceedings of the 6th Workshop on Data Management for End-To-End Machine Learning, DEEM 2022 - In conjunction with the 2022 ACM SIGMOD/PODS Conference
BT - Proceedings of the 6th Workshop on Data Management for End-To-End Machine Learning, DEEM 2022 - In conjunction with the 2022 ACM SIGMOD/PODS Conference
PB - Association for Computing Machinery, Inc
Y2 - 12 June 2022 through 12 June 2022
ER -