Graham began his teaching career in 1970 and taught in schools in Glasgow and Dunbartonshire.
He worked as a Curriculum Evaluator for the Consultative Committee on the Curriculum. During this period, he was seconded to BP to review links between education and industry. His report, 'Industry and Scottish Schools', was published in 1981.
Graham joined Her Majesty's Inspectorate of Education (HMIE) in 1983 and was HM Senior Chief Inspector of Education from 2002 to 2010. In that role, he was Chief Executive of HM Inspectorate of Education and Chief Professional Adviser to the Scottish Government on all aspects of education outside the university sector.
Graham is the current President of the Standing International Conference of Inspectorates (SICI), which has 29 member inspectorates from across Europe. He has also undertaken country reviews of education on behalf of the OECD.
Following his retirement from HMIE, Graham was asked by the Scottish Government to undertake a national review of teacher education in Scotland. The review was successfully published in January 2011.
Graham was made a Companion of the Most Honourable Order of the Bath (CB) for his services to education in the 2009 Queen's Birthday Honours List.