The Kimberley Indigenous Cognitive Assessment (KICA) was initially developed and validated as a culturally appropriate dementia screening tool for older Indigenous people living in the Kimberley. This paper describes the re-evaluation of the psychometric properties of the cognitive section (KICA-Cog) of this tool in two different populations, including a Northern Territory sample, and a larger population-based cohort from the Kimberley. In both populations, participants were evaluated on the KICA-Cog tool, and independently assessed by expert clinical raters blinded to the KICA scores, to determine validity and reliability of dementia diagnosis for both groups. Community consultation, feedback and education were integral parts of the research. for the Northern Territory sample, 52 participants were selected primarily through health services. Sensitivity was 82.4% and specificity was 87.5% for diagnosis of dementia, with area under the curve (AUC) of .95, based on a cut-off score of 31/32 of a possible 39. for the Kimberley sample, 363 participants from multiple communities formed part of a prevalence study of dementia. Sensitivity was 93.3% and specificity was 98.4% for a cut-off score of 33/34, with AUC = .98 (95% confidence interval: 0.97–0.99). There was no education bias found. The KICA-Cog appears to be most reliable at a cut-off of 33/39.