The club was established as Grays Juniors in 1890, before merging with former Southern League club Grays United to form Grays Athletic. The newly formed club joined the Grays & District League, later joining Division Two B of the South Essex League in 1908. They were runners-up in their first season in the division and won it in their second season, earning promotion to Division One for the 1909–10 season. The club were founder members of the Athenian League in 1912, but the league closed down in 1914 due to World War I and Grays transferred to the Premier Division (Amateur) of the London League. They won the division in their first season, with the reserves also winning Division One. The league was suspended at the end of the 1914–15 season, with Grays rejoining in the Premier Division when it restarted in 1919–20.
Grays were Premier Division runners-up in 1920–21, before winning the division in 1921–22. In 1924 the club joined the Kent League but after two seasons in mid-table, they returned to the London League in 1926. The return was a success as the club were Premier Division champions in 1926–27 and runners-up in the next two seasons, before winning the title again in 1929–30 After finishing as runners-up in 1930–31, the club spent most of the 1930s in mid-table, although they won the league's Challenge Cup in 1936–37.
After World War II Grays were founder members of the Corinthian League, and were its inaugural champions and League Cup winners in 1945–46. They were league runners-up in 1951–52, and the following season saw them qualify for the first round of the FA Cup for the first time, losing 5–0 at home to Llanelli. They were league runners-up again in 1954–55 and 1956–57, before rejoining the Athenian League in 1958. When the league gained two more divisions in 1963, the club were placed in the Premier Division, where they remained until being relegated to Division One at the end of the 1971–72 season, in which they had finished bottom of the table. In 1977 the league was reorganised into a single division, and the club were runners-up in 1982–83.
In 1983 Grays transferred to Division Two of the Isthmian League. League reorganisation saw them placed in Division Two South in 1984 and they went on to win the division at the first attempt, earning promotion to Division One. In 1987–88 they were Division One runners-up, resulting in promotion to the Premier Division.In 1988–89 the club reached the first round of the FA Cup for a second time, losing 2–0 at Bath City. They remained in the Premier Division until being relegated back to Division One at the end of the 1996–97 season. In 1999–2000 the club were Division One runners-up, earning promotion back to the Premier Division. In 2000–01 another appearance in the FA Cup first round saw them play Football League opposition for the first time, losing 4–0 at Reading.
A sixth-place finish in 2003–04 resulted in Grays being placed in the new Conference South for the 2004–05 season. They went on to win the new division in its first season, earning promotion to the Conference National. The season also saw them reach the final of the FA Trophy, in which they beat Hucknall Town 6–5 on penalties after a 1–1 draw at Villa Park. In their first season in the Conference National the club finished third, qualifying for the promotion play-offs. However, they lost 5–4 on aggregate to Halifax Town. They also reached the second round of the FA Cup, beating York City 3–0 at Bootham Crescent before losing 3–0 at Mansfield Town, as well as retaining the FA Trophy with a 2–0 win over Woking in the final. The club's league form subsequently faded and after narrowly avoiding relegation in 2006–07 and 2008–09, they finished bottom of the Conference National in 2009–10 and resigned from the Football Conference.
Grays were initially rejected by the Isthmian League and were lined up to play in the Essex Senior League. However, an appeal to the Football Association saw them placed in Division One North of the Isthmian League. A fifth-place finish in 2011–12 saw them qualify for the play-offs, in which Enfield Town defeated them 3–1 on penalties after a 2–2 draw in the semi-final. However, the club were Division One North champions the following season with a club record 102 points, earning promotion back to the Premier Division.
In August 2016 the club became community- Supporter owned They went on to finish bottom of the Premier Division in the 2016–17 and were relegated back to the (renamed) North Division.