The nominal scientific method dissevers a discovery from the experiment that made it as early and thoroughly as possible—by suppressing the personalities of the experimenters, by using standardized equipment and methods of analysis, by ensuring reproducibility and undertaking it. All these mechanisms are relatively weak in cognitive psychology. Success results in advancement, experiments themselves with their individual variability are the instruments, and the expense and inconvenience of recruitment, with the backlog of experiments yet to be done, make reproducing experiments a last resort. But psychology has never worked the nominal way—try it, and you get Behaviorism—and this way seems, for the most part, to work.
Cognitive psychology had to fight hard to free itself from Behaviorism. Here we come to the problem. Behaviorism rejected introspection; cognitive psychology accept it; but it exchanges uncritical rejection for uncritical acceptance. I think that there is something that cognitive psychology gets wrong about introspection: it assumes that our perceptions resemble our perception of our perceptions.
(Nominally cognitive psychology rejects "introspection" but accepts "self-reporting"—I do not understand the difference.)
For example: a form of experiment is to present a list of words on a common theme for memorization. The list omits some particularly obvious entry that could be expected to occur with the rest. When memorizers repeat the list back, they often supply this absent word. The obvious conclusion and the one that cognitive psychology draws is that memorizers have perceived the absent word; that a subconscious process—subconscious in the sense of neurologically substratal—inserts the extra word into the memorizer's perception of the list.
But this may not be so. Precisely because the list is simple it obscures the distinction between perception and perception-of-perception. It is also a plausible interpretation that memorizers do not perceive the extra word; instead, in reproducing the list, the memorizers perceive their own perception of the list in an incorrect yet conscious way. This multiplies entities, but I think it provides more parsimonious explanations.