I feared the GFA would interfere with my job – Kwesi Appiah