Jerusalem was the site of Christ's crucifixion and resurrection, and as such it was -- from the beginning of Christianity -- the Holy City. No city in Christendom, not even Rome, was considered so sacred as Jerusalem. The loss of Jerusalem to a Muslim army in 638 was a blow to all Christians, and the desire to regain control of Jerusalem was an overt or a latent goal of all subsequent campaigns against Islam. That it took 461 years was a function of inadequate capacity not lack of will. Below is a brief summary of why Jerusalem was in the minds of medieval Christians so much a part of their heritage.
From the time of Christ's crucifixion onwards, a small Christian community lived in the city. Admittedly, Jerusalem remained a predominantly Jewish city, despite the destruction of the Second Temple in 70 AD, until the Romans expelled the entire Jewish population after renewed insurrection in 135.
Jerusalem was then rebuilt by Hadrian, given a new name (Aelia Capitolina) and Roman temples were built on the site of the old Jewish Temple and on the sites sacred to Christians. The objective was to humiliate Jews and Christians alike and, in the case of the Christians, to wipe out the association with Christ's life and passion at certain sites. Furthermore, both Jews and Christians were expelled from Jerusalem and persecuted. Aelia Capitolina was a pagan city, and as such it was nothing more than provincial backwater of little importance to the Roman Empire.
All that changed after the Emperor Constantine came to the Imperial throne. His mother, Helena, was Christian. She persuaded her son to end the persecution of Christians in 313 and to allow her to make a pilgrimage to Palestine. She is credited with locating the sites of Christ's nativity, execution and resurrection. Little more than a decade later, a massive construction project was undertaken to turn Jerusalem into a major Christian capital. In 326 work began on two magnificent basilica: one in Bethlehem over the site of the nativity and the other in Jerusalem over the site of Christ's grave (the Holy Sepulcher).
For nearly 300 years thereafter, Jerusalem was one of the most important cities of the Eastern Roman Empire. Although unable to compete with Constantinople and Alexandria in terms of trade and industry, it was revered for its sacred traditions. Pilgrims flooded to the sacred sites providing a strong economic base that was reflected in the construction of churches, monasteries, workshops, markets, inns and residences. The inhabitants of this revitalized city were primarily Christian, although Jews were allowed to return as well. The population exceeded 60,000 -- a very substantial population for this period.
In 614 disaster struck. A Persian army surrounded Jerusalem and took it after a 21 day siege. Aided by Jewish allies, the Persians slaughtered an estimated 26,500 Christian inhabitants and enslaved an additional 35,000. The Church of the Holy Sepulcher was raised to the ground. In an ironic twist, the Church of the Nativity in nearby Bethlehem escaped destruction because the mosaic Adoration of the Magi over the portal depicted the Magi as Persian kings; the Persian troops stayed their hand out of respect for the "Persian" kings.
Thirteen years later in 627, Emperor Herakleios wrested control of Jerusalem back from the Persians after defeating them decisively at the Battle of Ninveh. The treaty following the battle required the Persians to withdraw from all conquered territories, including Palestine and so Jerusalem. Yet while Byzantine control over Jerusalem was thus restored, the destruction of the city's sacred monuments and the slaughter or enslavement of the inhabitants could not be so easily overcome. All the Emperor could do was start a rebuilding and resettlement program. In punishment for their role in the slaughter and destruction of the Christian population thirteen years earlier, however, the Jews were again expelled from Jerusalem and prohibited from entering.
Thus at the time of the Muslim conquest, the city was exclusively Christian. Furthermore, the fact that despite the terrible losses and destruction, the city held out for a whole year before surrendering to the armies of Caliph Omar I is a testimony to how vigorously the Christian defenders resisted the Muslim attack. In the end, they were too weak -- as was the entire Eastern Roman Empire.
It would be 461 years before Jerusalem was restored to Christian control, but that was not because Jerusalem was forgotten or unimportant, but rather because Christendom had been fighting for its very existence against an expansionary Islam from Spain and Italy to Constantinople, Cyprus and Armenia. See: Genesis of the Crusades.