How to force a wordbreaker to be used in Sharepoint Search

I had a requirement to crawl Japanese content and we had an English Install of Sharepoint 2007 (MOSS). Now for some reason the locale was not being identified by the crawler for the HTML (i.e. all the Publishing pages) as 1041 (i.e. the Japanese locale), and the default English (US) word breaker was used. I wanted to force the Japanese wordbreaker and finally found that I need to make a registry change (*** PLEASE NOTE REGISTRY CHANGE IS NOT A GOOD IDEA, BEFORE DOING ANY REGISTRY CHANGE PLEASE TAKE A BACKUP OF YOUR REGISTRY***).

To force the crawler to use the JAPANESE WORD BREAKER, I changed the following registry key to use the Japanese WBreakerClass (This change is required on your Index Server):

 

*********************************************************************************************************************************************************************

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office Server\12.0\Search\Setup\ContentIndexCommon\LanguageResources\Default\English (United States)]           

"WBreakerClass"="{E1E8F15E-8BEC-45df-83BF-50FF84D0CAB5}"                                                                                                                                                                

*********************************************************************************************************************************************************************

 

Now the crawler uses Japanese WBreakerClass instead of English WBreakerClass and all my content is crawled properly as well as the Search results are working great with proper hithighlighting (which was not working earlier).

Please note that by only forcing Japanese wordbreaker even for English content has a side effect such as breaking definition extraction.