As we may all know, ligatures and diæreses have long become obsolescent. However, I see the logic behind spelling words with ligatures and diæreses. For example: algæ, formulæ, æon, æqulateral, æternal, œuvre, œsophagus, fœderal, coöperation, aëroplane, etc.
Is it acceptable that I spell similar words as such (i.e. in modern use: papers, theses, publications)? Are there any disadvantages or drawbacks?
If I do spell these words with ligatures and diæreses, does it mean that I have to use obsolete spellings for all the words or I can choose?
If spellings such as paediatric, foetus, anaesthesia are encouraged in BrE (and medical/formal contexts), does it mean that all the other words should be spelt with ae or oe as well?
Ligatures and diaereses are not generally used in modern English text. However, whether or not they are “acceptable” depends on many factors.
The easiest way to judge if something is acceptable is if you have an institutional style guide that you’re supposed to follow. Any reasonably complete style guide should cover this topic. I believe the most common practice in formal contexts is to only use ligatures and diaereses in words from foreign languages, not including Greek or Latin (so œuvre might be spelled with a ligature, and Noël might be spelled with a diaeresis) and in official names such as Encyclopædia Britannica. The New Yorker still uses the diaeresis to indicate hiatus in some English words such as coöperation, but this is unusual and perceived as quaint. I don’t think even they would use a diaeresis in aëroplane since the second vowel in this word is elided for nearly all speakers.
In words from Latin or Greek, the ligatures æ œ will be perceived as stylistic variants of the digraphs ae oe. These ligatures are not generally used in modern typography.
Words that can’t have a ligature because they are spelled with e
Several of the words you listed in your original question are never spelled with a ligature or a digraph in Modern English. Æquilateral/aequilateral, æternal/aeternal, fœderal/foederal are all entirely obsolete. These words are always spelled equilateral, eternal, federal. You can find out which spellings are in general use by looking up the word in a dictionary.
All modern English is inconsistent in this regard: while the specifics vary, both Brits and Americans can only have ae/oe in some words, and can only have e in others. You should not try to spell consistently based on the etymology: there is no way to do this without using non-standard spellings.
Words where you can choose between e and ae/oe
You only have a choice for some words. For example, æon and œsophagus may be written with just e or with a digraph (rendering these digraphs as ligatures would not be usual style, but it would not look too strange in my opinion). In some cases, one choice is standard for a particular variety of English (for example, oesophagus is standard in British English but not in American English).