against England in Cardiff. He turned professional in February 1928 when he joined West Bromwich Albion as a 17-year-old.
Murphy made his debut in a 1–0 defeat away to Blackpool on 5 March 1930 and played one further league game during his first season. In the following season, 1930–31, West Bromwich Albion won the FA Cup and promotion from the Second Division, but Murphy had yet to establish himself in the team and again made just two appearances. He became a regular in the Albion side upon the club's return to the First Division; from 1931–32 to 1934–35 he appeared 149 times in league and cup, helping his team to achieve four consecutive top ten finishes, including fourth place in 1932–33. The 1934–35 season saw Murphy miss just one match all season, and he helped Albion to reach the 1935 FA Cup Final, which they lost 4–2 to Sheffield Wednesday.

Murphy played more than 200 times for Albion, before moving to Swindon Town in 1939, but the Second World War ended his spell at Swindon almost as soon as it had begun. Murphy was also called up to the Welsh national team during the 1930s, winning 15 caps.

Managerial career
During the Second World War, Murphy was giving a speech about football to a band of troops, and in attendance was Matt Busby. Busby was so impressed by Murphy's speech that, upon his appointment as manager of Manchester United, he made Murphy the first signing of his tenure at the club.

Murphy had the role of "chief coach" from 1946 until 1955, and became assistant manager in 1955 after Manchester United won their third FA Youth Cup in a row. It was Murphy's responsibility at the club to train the young footballers who were to become the "Busby Babes", which included Duncan Edwards and Bobby Charlton. Prior to this the larger teams had mainly bought rather than developed their players but instead Busby decided to slowly replace the older and more experienced players in his team with their youth players.

After the Munich air disaster of 6 February 1958, he temporarily took over as manager while Matt Busby recovered from his injuries and, having assembled a substitute team, steered United to the 1958 FA Cup Final. Murphy had not been on the fatal flight because he had been away managing the Welsh team in a World Cup qualifying game. Murphy managed Wales at the 1958 FIFA World Cup in Sweden when they reached the quarter-finals in their only appearance to date in a World Cup finals tournament. They lost 1–0 to Brazil (the eventual winners), to a goal by 17 year-old Pelé.