Below a repertory of a few websites that can be of use when you settle in or renovate your new home:
Notaires de France: http://www.notaires.fr/en
The website is in English and allows you to find a notary in your region but also give you some feedback about the French property market.
La Direction générale des Finances Publiques: http://www.impots.gouv.fr/portal/static/
The website is in French but allows you to create your account and pay your Taxe d'Habitation and Taxe Foncière online.
ErDF if you need a new electricity connection: http://www.erdf.fr/particulier 
GrDF if you need a new gas connection: http://www.grdf.fr
EDF the largest supplier of electricity in France: http://www.edfenergy.com
The website is in English and allows you to create an account online.
GDF Suez Dolce Vita the largest dual supplier of Gas and Electricity in France: https://www.gdfsuez-dolcevita.fr.
You can also create an account online but the site is in French only.
Energie-Info is a governemental website allowing you to find alternative suppliers of electricity and gas in your area http://calculettes.energie-info.fr/pratique/liste-des-fournisseurs.
France Telecom / Orange if you need to install a new phone line: http://www.orange.fr
The website allows you to create an online account.
These are just the principal organisations you will need to know about when buying or renovating a property but if you need to get in touch with other organisations such as Town Halls, the public Health Service or the RAM (if you are self-employed)...just get in touch with me and I will happily contact them on your behalf.