Buspirone is one of those drugs that looks good on paper but is actually quite weak in practice. It's like moclobemide: it seems promising but in practice it's very weak. It's rated at 6.5/10 for anxiety on drugs.com: https://www.drugs.com/comments/buspirone/buspar.html
I think ashwagandha decreases 5-HT1A activity and partial-agonism is more-or-so the same - a net decrease in activity. From wikipedia:
when both a full agonist and partial agonist are present, the partial agonist actually acts as a competitive antagonist, competing with the full agonist for receptor occupancy and producing a net decrease in the receptor activation observed with the full agonist alone. [font=sans-serif]Clinically, partial agonists can be used to activate receptors to give a desired submaximal response when inadequate amounts of the endogenous ligand are present, or they can reduce the overstimulation of receptors when excess amounts of the endogenous ligand are present.[/font]
https://en.wikipedia.org/wiki/Partial_agonist
So in short, if you have sufficient serotonin already then buspar will act as an antagonist, but if you're low on serotonin then buspar can act as an agonist. Does ashwagandha antagonize the 5-HT1A receptor or does it just decrease serotonin while agonizing the 5-HT2A receptor?
Ultimately though, you have to take buspar to see if it's good for your or not. The only annoying thing about it is its short half-life meaning you have to take it 3x a day. if you do take it, let us know how it works out for you.