Yes, you can refuse the CDF; however, HMRC may pursue a criminal investigation if they believe serious fraud has taken place. Seeking expert advice before making a decision is highly recommended.