Researchers have long viewed patterns of species association as key to understanding the processes that structure communities. Community-level tests of species association have received the most attention; however, pairwise species associations may offer greater opportunity for linking patterns to specific mechanisms. Although several tests of pairwise association have been developed, there remain gaps in our understanding of their performance. Consequently, it is unclear whether these methods reliably detect patterns of association, or if any one method is superior. We maximized association patterns for single species pairs in synthetic community matrices and examined how accurately five pairwise association tests found that pair, while not finding others (i.e., type I and II error rates). All tests are more likely to miss patterns of association than to falsely detect them. When we maximized association for a species pair that included one or more rare or common species, tests were frequently unable to identify that pair as significantly associated. Consequently, these tests are best suited for identifying significant associations between pairs of species that occur in an intermediate number of samples; for such pairs, three of the five tests considered here detected 100% of the pairs for which we maximized associations.