High-performance computing as significantly evolves during the last two decades with the advent and consideration of multicores configurations and general-purpose GPUs. Cutting-edge AI, which typically involving large models and huge volume of training datasets, cannot be considered without the support of HPC infrastructures and techniques.