This will not save you power. The app_timer use RTC1 for time keeping. Most likely, you would only generate more work for yourself and risk introducing some bugs because you fail to account for some things that we do in the app_timer implementation.
That said, there is no problem using the RTC directly if you really want. If you are using the nRF5 SDK, you can refer to the Real Time Counter Example.
