The current weakest link in my thesis is this (IMO): how to connect merely having beliefs about mathematics that help us solve problems, and yield correct applications to concrete situations to having beliefs about mathematics that are reasonably reliable.
Couldn't totally false mathematical theories nonetheless be perfectly correct with regard to their concrete applications?
Also, even if our beliefs would indeed perfectly accurately describe some concrete objects, how can we count as refering to these objects, given that we have no causal contact with them?
My current best answer is this:
Think of human mathematicians as observing certain regularities (e.g. whenever there are 2 male rhymes and 2 female rhymes in a poem there are at least 4 rhymes all together), and then positing mathematical objects "the numbers" whose relationship to one another is supposed to echo these logical facts.
(This is a reasonable comparison because what we actually do is like this, in that we happily make inferences from a proof that "a+b=c" to the expectation that when there are a male rhymes and b female rhymes there are c rhymes all together. We behave as though we know there's this relationship between the numbers and logical facts, so it's not too much of a stretch to compare us to people who actually consciously posit that there is some collection of abstract objects whose features echo the relevant logical facts in this way.)
Now either there are abstract objects or not.
If there aren't abstracta (as the fictionalist thinks), the fact that mathematicians only care about structures makes it plausible to think of them as talking about the fiction in which there are such objects.
Thus, our abstract-object positing mathematicians will count as speaking about the fiction in which there are objects whose features echo the logical facts about addition in the intended way. They will also count as knowing lots of things about what's true in this fiction.
Also, note that insofar as these mathematicians propose new things that "intuitively must be true of the numbers" their intuitions will be disciplined and corrected by the fact that the relevant applications are expected, so there's a systematic force which will keep some degree of match between their claims about this fiction and what's actually true in this fiction.
If there are abstracta, then there are abstract objects with many different structures, in particular structures corresponding to every consistent first order theory (note this is even true if the only mathematical objects there are are sets! the completeness theorem guarantees that there are models of every such theory within the heirarchy sets). So there will be some collection of objects whose features match those expected by our positers (note that the positers only really care about structural features of "the numbers" not whether they are fundamental mathematical objects etc).
Now, how can our positers count as referring to some such objects? Well, as noted above, we have systematic mechanisms of belief revision which kick back and insure that their claims about the numbers must match with logical facts, and hence with the real facts about these collections of suitable abstracta. Just as looking at llamas helps ensure that certain kinds of false beliefs about llamas which you might form would be corrected, applying arithmetic insures that certain kinds of false general beliefs you might form about the numbers would be corrected (those which lead to false consequences about sums).
Thus, we have a situation where people not only have many beliefs that are true about the numbers, and the tendency to make many truth-preserving inferences, but also where these beliefs have a certain amount of modal stability (many kinds of false beliefs would tend to be corrected). Even Fodor thinks that making correct inferences with or is sufficient to allow or to make the right kind of contribution to the truth value of your sentences, so why should the same thing not apply to talk about numbers, given that we now have not only many good inferences but this kind of mechanism of correction which improves the fit between our beliefs about the numbers and the numbers?
You might still worry that there will be so many mathematical objects which have all the features which we expect the numbers to have - how can we count as referring to any one such structure, given that our use fits all of them equally well? And if we don't uniquely pick out a structure, how can our words count as refering and being meaningful? But note that to the extent that our use of the word "the numbers" is somehow ambiguous between e.g. different collections of sets, our use of the word "human bodies" would seem to be equally ambiguous between e.g. open vs. closed sets of spacetime points. So either meaningfully talking about objects is compatible with some amount of ambiguity, or the above kind of reasoning doesn't suffice to establish ambiguity.