The World Series as we know it began in 1903, when the Boston Americans beat the Pittsburgh Pirates 5-3 in a best-of-nine series. There were some postseason series before that, though.
From 1884-1890, the champions of the National League and American Association met for series of various lengths. Baseball-reference now has the stats from those series.
The earliest was the 1884 series between the Providence Grays and the New York Metropolitans, which was swept by Providence 3-0. Old Hoss Radbourn, fresh off a 59-12 regular season, pitched all three games for the Grays and didn’t allow a run in 22 innings. Fellow Hall of Famer Tim Keefe was on the losing side of two of those games. Jerry Denny of the Grays was the star hitter of the series, collecting the only homer.
There was also one more “Championship Series” in 1892, one year after the American Association folded. That featured the National League’s first- and second-half champions, with the Boston Beaneaters defeating the Cleveland Spiders 5-0-1. Three Hall of Famers pitched in that one, with Kid Nichols going 2-0 for Boston and Cy Young and John Clarkson both losing twice for Cleveland. Hugh Duffy, the Hall of Fame outfielder, hit .462 and drove in nine runs for the Beaneaters.
It’s worth looking at the rest if you have some time to kill.