Contrary to the current regulatory trend regarding extreme risks, the purpose of this paper is to emphasize the necessity of considering the Value-at-Risk (VaR) with extreme confidence levels like 99.9%, as an alternative way of measuring risks in the "extreme tail". Although the mathematical definition of the extreme VaR is trivial, its computation is challenging in practice, because the uncertainty of the extreme VaR may not be negligible for a finite amount of data. We begin to build confidence intervals around the unknown VaR. We build them using two different approaches, the first one uses the asymptotic Gaussian result and the second saddlepoint approach, the latter proves to be more robust when we use finite samples. We compare our approach with other methodologies which are based on bootstrapping techniques, focusing on the estimation of the extreme quantiles of a distribution. Finally, we apply these confidence intervals to perform a stress testing exercice with historical stock returns during the financial crisis, in order to identify potential violations of the VaR during periods of turmoil on financial markets.