Exhibit 23.1

Consent of Ernst & Young LLP, Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement (Form S-3 No. 333-24135) pertaining to the Palm Harbor Homes, Inc. Employee Savings Plan, and in the Registration Statements (Form S-3 No. 333-52535 and Form S-3 No. 333-118669) and in the related Prospectuses of Palm Harbor Homes, Inc. and subsidiaries, of our reports dated May 23, 2006, with respect to the consolidated financial statements of Palm Harbor Homes, Inc., Palm Harbor Homes, Inc. management’s assessment of internal control over financial reporting, and the effectiveness of internal control over financial reporting of Palm Harbor Homes, Inc. included in this Annual Report (Form 10-K) for the year ended March 31, 2006, filed with the Securities and Exchange Commission.

/s/    ERNST & YOUNG LLP

Dallas, Texas

June 6, 2006