The current Library traces its roots to its establishment as the Maryland State Library by the General Assembly in 1826 Md. Laws, Chapter 53, passed on February 2, 1827. The Library opened its doors in the State House for the first time in December 1827. After serving as a repository for a wide variety of topics, its collection began to focus on legal materials by the late 1800's. In 1892, the Library became governed by a committee of appellate judges; twelve years later it moved into the then-new Court of Appeals Building. The Library moved to its current location in 1972. Renamed the State Law Library, it officially came under the Judicial Branch in 1978 (Chapter 128, Laws of 1978). The legal authority for the State's oldest law library appears in §§ 13-501 to 13-504 of the Courts & Judicial Proceedings Article of the Annotated Code of Maryland.

During much of the 20th Century, the Library was headed by just two library directors. Mr. Nelson Molter served as Cataloger from 1932 to 1939, as Law Librarian from 1939 to 1955, and then as Director of the State Library from 1955 to 1977. (The position of State Librarian was a political appointment according to the State Constitution until 1959. Because this appointee seldom had much contact with Library operations, Mr. Molter was responsible for the daily management of the Library throughout his tenure.) Michael S. Miller, a recipient of the American Association of Law Libraries' highest honor, the Marian Gould Gallagher Distinguished Service Award, served as Director from 1977 to 2005. Steven P. Anderson, also a Gallagher recipient, served as the Library Director from 2005 to 2021. 

In 2019, the State Law Library was renamed as the Thurgood Marshall State Law Library (Chapters 38 and 39, Laws of 2019).

The Library retains in its Special Collection Room historic circulation records and printed library catalogs from the 19th Century. Another source of information about the Library's history is Robert W. Coover's "A History of the Maryland State Library 1827-1939 (with a summary of events from 1939-1959)."