Last week I decided that it was time to upgrade my Linux servers from Ubuntu 14.04 LTS to 16.04 LTS. This was needed since the 14.04 LTS already disabled support for many hardware enablement stacks – all but one – and many newer software implementations are only available in the latest Ubuntu LTS release. The same thing goes for the Linux Mint 17 vs 18, the 18 version uses the same kernel and software repositories than the Ubuntu 16.04 LTS release. Little did I know that doing these upgrades would become a nightmare since many of the AMD GPU cards that I use for back-testing stopped working. On today’s post I will talk a bit about this experience and what I did to get most of my cards to work on the new Ubuntu LTS release.
–
–
After completing the smooth update of the operating systems to the latest version I was surprised to see that the drivers that I used to use in Linux for AMD cards, called fglrx, are now unsupported in the latest versions of Ubuntu. This meant that none of my cards worked and I couldn’t continue using them for back-testing. In particular I have R9 290x and R9 280x cards which I need to use in order to do mining. My first attempt to get this to work was to use the Radeon open source drivers that come with Ubuntu and I was happy to see that my cards were immediately recognized by the OpenCL mesa icd. However I was immediately disappointed to see that these drivers only support OpenCL 1.1 for these cards, which is a very old implementation that does not support some of the features that we use in our openCL kernels for back-testing. So this was a deal breaker for the Readeon drivers.
The next thing I did was to attempt to use the proprietary amdgpu drivers that AMD has been working on to replace the previous fglrx drivers. I downloaded the drivers from here and proceeded with the driver installation. After this the R9 290x cards worked but the speeds were dramatically slow – ultra slow – with execution times that were anywhere from 2 to 10x slower than what I had when using the fglrx drivers. In general I was doing back-testing at around 0.0300-0.0800 ms per system and now times were in the order of 0.2-0.3 ms per system, this implies that my mining would be tremendously slow, something that is unacceptable given that we perform billions of backtests each day. The R9-280x card, which uses a Tahiti instead of a Hawaii chip, did not work at all.
–
if test "$CONFIG_LOCALVERSION_AUTO" = "y"; then
# full scm version string
res="$res$(scm_version)"
else
# append a plus sign if the repository is not in a clean
# annotated or signed tagged state (as git describe only
# looks at signed or annotated tags - git tag -a/-s) and
# LOCALVERSION= is not specified
#if test "${LOCALVERSION+set}" != "set"; then
# scm=$(scm_version --short)
# res="$res${scm:++}"
# fi
fi
–
I read that there were some special AMD kernel support options that I could use in order to improve the performance of the cards so I decided to try rebuilding the linux kernel. For this I followed the instructions here which allowed me to rebuild a 4.4 kernel using the CONFIG_DRM_AMDGPU_CIK=Y option through a very simple process. You can simply edit the .config file in the root folder of your kernel and change the value for this. I also advice against using kernels higher than 4.4 as I ran into other problems with newer kernels. For example when I tried using 4.8 my keyboard did not even work (luckily I also had an emergency generic keyboard that would work so that I could change my kernel again). If you build this kernel also bear in mind that the kernel name may contain the word “custom” plus a “+” in the end which some programs really hate (dropbox would not work as it would crash because it couldn’t parse the kernel version). To remove this you can go to the scripts/setlocalversion script in your kernel folder and comment out the lines in the end of the file as showed above. Also make sure you set CONFIG_LOCALVERSION_AUTO=N in your kernel .config to ensure that the above works. Also modify the build.sh as shown below.
By enabling the CIK support in the kernel I was now able to get my AMD R9 290x cards to work very well. As a matter of fact mining speeds were increased very significantly with mining times now being around 0.0100-0.0200 ms/system instead of the previous 0.0300-0.0800. Sadly I keep almost the exact same mining power as before because although I am mining almost 2x faster than before I have several R9-280X cards that are yet to be supported by the AMDGPU drivers and hence I can only use half of my GPU power for back-testing. Luckily this increase in speed for the R9-290x cards means that I won’t have a decrease in productivity because of some of my cards being idle and my backtesting/power efficiency did double as I now only need to power half the cards to do the same job. Hopefully support for Tahiti cards will come and I will be able to use my full array of GPU cards at a higher efficiency. Interestingly – as you can see above – the old drivers said the card had 44 cores while the new drivers say there are only 14.
–
#!/bin/bash
#GitHub This: https://github.com/Turbine1991/build_ubuntu_kernel_wastedcores
#GitHub WastedCores: https://github.com/jplozi/wastedcores
##Setup
BUILD_PREFIX=""
##Retrieve/increment build value
##BUILD_COUNT=$(cat ".build_count")
##((BUILD_COUNT++))
##echo "$BUILD_COUNT" > .build_count
#
##Declare
DEB_FILE="$BUILD_PREFIX"
cd "kernel"
(
##Build into .deb package
cd "mainline-crack"
##Experiment with some optimizations
#export CFLAGS=' -march=native -mtune=native -mcpu=native -Ofast -fwhole-program -fmodulo-sched -fmodulo-sched-allow-regmoves ' \
# && export CXXFLAGS=' -march=native -mtune=native -mcpu=native -Ofast -fwhole-program' \
# && export LDFLAGS=' -fwhole-program '
#
#if [[ ! -f .optimized ]]; then
#Put optimizations into Makefile
# sed -i '/HOSTCFLAGS =/c\HOSTCFLAGS = -march=native -mtune=native -Ofast -fmodulo-sched -fmodulo-sched-allow-regmoves -fno-tree-vectorize -std=gnu89' Makefile
# sed -i '/HOSTCXXFLAGS =/c\HOSTCXXFLAGS = -march=native -mtune=native -Ofast' Makefile
# touch .optimized
#fi
make clean && fakeroot make-kpkg -j`nproc` --initrd kernel_image kernel_headers
echo "Everything's Complete"
)
##Request and attempt installation of compiled packages
if ls *$DEB_FILE*.deb 1> /dev/null 2>&1; then
read -p "Install kernel (y/n): " -n 1
if [[ $REPLY =~ ^[Yy]$ ]]; then
dpkg -i *$DEB_FILE*.deb
echo "Please Reboot"
fi
fi
–
If you want to upgrade to Ubuntu 16.04, Mint 18 or any release that uses a 4.4 kernel please bear in mind that the old AMD fglrx drivers won’t work and therefore you might not be able to work with your GPU cards. This is especially true if you require OpenCL 1.2 for the software you will be using with your cards. If you have Tahiti cards or are running any other GCN 1.0 cards bear in mind that they are not supported by the drivers yet (although some of them might in the future). Also bear in mind that recompiling your kernel with the proper options is a MUST since otherwise your cards might run super-slow, even if they are compatible with the latest drivers. If you would like to learn more about using GPUs for back-testing and how you too can use this amazing technology to explore very large price action based or machine learning system spaces please consider joining Asirikuy.com, a website filled with educational videos, trading systems, development and a sound, honest and transparent approach towards automated trading.strategies
