EVERYONE has the intuition that some languages are more difficult than others. For the native English-speaker, professional agencies that teach foreign languages have made it quite clear. America’s state department reckons that Spanish, Swedish or French can be learned in 575-600 class hours (“Category 1”). Russian, Hebrew and Icelandic are more difficult (1100 class hours, “Category 2”). And Arabic, Japanese, Mandarin and a few others are in the hardest group, Category 3, requiring 2200 class hours. But what makes a language difficult? 
How long it takes to learn a language does not answer which ones are hard independent of the learner’s first language (nor the related question “How hard is English?”) Ranking languages on a universal scale of difficulty is itself difficult and controversial. Some languages proliferate endings on verbs and nouns, like Latin and Russian. Such inflection can be hard for learners who are not used to it. Several years ago, two scholars found that smaller languages (those with less contact with other languages) tended to have more inflection than big ones. By contrast, creole languages—which arise between groups that do not share a common language—are thought by scholars to be systematically simpler than other languages, even after they become “normal” languages with native speakers. They typically lack heavy inflection.
But inflection is only one element of “hardness”. Some languages have simple sound systems (such as the Polynesian languages). Others have a wide variety of sounds, including rare ones that outsiders find hard to learn (like the languages of the Caucasus). Some languages (like English) lack or mostly lack grammatical gender. Some have dozens of genders (also known as “noun classes”) that must be learned for each noun. Languages can have rigidly fixed or flexible word order. They can put verbs before objects or even objects before subjects. Yet it is not clear how to rank the relative difficulty of exotic consonants, dozens of genders or heavy inflection. Another recent approach sought to go around the problem by finding languages that had the most unusual features, skirting the question of whether those features were “hard”. Comparing 21 feature parameters across hundreds of languages, they ranked 239 languages. Chalcatongo Mixtec, spoken in Mexico, was the weirdest. English came in place number 33. Basque, Hungarian, Hindi and Cantonese ranked as among the most “normal”. The researchers did not find any larger similarities between “weird” and “normal” languages. (For example, they do not claim that smaller or bigger languages tend to be “weirder”.) But again, the caveat is that this only compares which languages are unusual in a global context, not which are hard.
So the two most robust findings seem to be that smaller languages are more heavily inflected, and that languages farther from your own in the linguistic family tree will be harder for you to learn. If you want a challenge, a good bet is to pick a tiny language from halfway around the world.