This list is actually compiled for a friend of mine, but, hey, it needs to go somewhere and this is as good a place as any. I don't know if anyone else is interested, but she's a fellow Austen fan. This is basically some of the context (1780-1830), focussing especially on the French Revolution and the gothic.
Some Misc. Critical Works:
Abrams, M. H. Natural Supernaturalism: Tradition and Revolution in Romantic Literature (New York: Norton, 1971).
Butler, Marilyn, Romantics, Rebels and Revolutionaries: English Literature and Its Background (Oxford University Press, 1981).
Clery, E. J. Women's Gothic from Clara Reeve to Mary Shelley (Manchester: Writers and Their Work, 2000).
Jones, Chris, Radical Sensibility: Literature and Ideas in the 1790s (London, 1993).
McCalman, Ian et al, eds., The Oxford Companion to Romanticism and Revolution (Oxford University Press, 1999). [This is excellent but far too expensive to buy. I don't know if you have any decent libraries near you.]
Influential Women Poets
Ashfield, Andrew, ed., Romantic Women Poets 1770-1838: An Anthology (Manchester University Press, 1995).
Curran, Stuart, ed., The Poems of Charlotte Smith (New York, 1993).
Kelly, Gary, ed., Felicia Hemans, Selected Poems (Peterborough, ON: Broadview Press, 2002).
Pascoe, Judith, ed., Mary Robinson: Selected Poems (Ontario: Broadview Press, 2000).
Wu, Duncan, ed., Romantic Women Poets: An Anthology (Oxford: Blackwell, 1997).
There's also that biography of Robinson that I told you about, very readable and enjoyable:
Byrne, Paula, Perdita: The Life of Mary Robinson (Suffolk: Harper Perennial, 2005).
And don't forget Ann Radcliffe's The Italian (or similar gothic novel) for Northanger Abbey.
As for the Revolution...
I'd recommend reading some of what was written about it at the time. There's Edmund Burke's famous Reflections on the Revolution in France (1791). And, for the other side of the argument: The Rights of Man by Thomas Paine; Vindication of the Rights of Men by Mary Woolstonecraft and Impartial Reflections on the Present Situation of the Queen of France by a Friend to Humanity by Mary Robinson.
Much of this is online.
Other fantastic web resources:
Romanticism on the Net: http://bbk.ac.uk/english/ac/wrew.htm
University of Virginia e-texts: http://etext.lib.virginia.edu.modeng/modeng0.browse.html
The French Revolution: http://www.wwnorton.com/college/english/nael/romantic/topic_3/welcome.htm
And you'd love The Republic of Pemberley.
Books, though, can be cheaper than you think. There's eBay and Amazon, of course, but also Blackwells and Play.com where you don't have to pay postage (in the UK, at least - I don't know about the US).