Why do languages have the words they do instead of some other set of words? In the first part of this thesis, I argue that cognitive and communicative demands strongly influence the structure of the lexicons of natural languages. It is known that words in natural language are distributed such that shorter words are more frequent and occur after more predictive contexts. I provide evidence that, at least in part, this pattern is driven by word shortenings (i.e., chimp → chimpanzee) and that word shortenings can be predicted by principles of efficient communication. I also show that, using nonce words with no pre-existing semantic meaning, a Zipfian correlation between length and frequency emerges in freely produced text and that this correlation is driven by participants’ tendency to reuse short words more readily than longer words. In addition to word length, I investigate phonetic probability in a corpus of 97 languages. Across a wide variety of languages and language families, phonetic forms are optimized for efficient communication. And, using baseline phonetic models, I show that the words in the lexicons of four languages (English, Dutch, German, and French) are more tightly clustered in phonetic space than would be suggested by chance alone. This thesis depends on standard methods in language research. How reliable is the data that we work with as a field? In the second part of this thesis, I tackle that question by examining two dominant methods in modern language research: behavioral experiments (specifically syntactic priming) and linguistic acceptability judgments. I present data, based on large-scale surveys, showing that many of the standard syntactic and semantic judgments in a mainstream linguistic journal are flawed. Using this data, I construct a Bayesian prior over judgments and give recommendations for performing small sample-size experiments in linguistics that will not overly burden researchers. Finally, I present a large-scale meta-analysis of syntactic priming (the largest meta-analysis of a psycholinguistic phenomenon) and find that, while many priming studies are severaly underpowered, there is no evidence of intense p-hacking.