Loans and Allowance for Loan Losses (Details 4) (USD $)
In Thousands, unless otherwise specified
Sep. 30, 2012
Dec. 31, 2011
Schedule of carrying amount of impaired loans    
Ending balance $ 369,670 $ 408,138
Construction, land development and other land loans [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 30,949 40,311
Secured by 1-4 family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 74,390 84,223
Secured by multi-family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 24,305 26,722
Secured by non-farm, non-residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 132,980 147,518
Commercial and industrial [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 43,255 43,060
Impaired loans with no related allowance recorded [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 26,297 34,586
Unpaid Principal Balance 26,297 34,586
Related Allowances      
Impaired loans with no related allowance recorded [Member] | Construction, land development and other land loans [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 3,867 7,005
Unpaid Principal Balance 3,867 7,005
Related Allowances      
Impaired loans with no related allowance recorded [Member] | Secured by 1-4 family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 208 186
Unpaid Principal Balance 208 186
Related Allowances      
Impaired loans with no related allowance recorded [Member] | Secured by multi-family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 2,694 2,884
Unpaid Principal Balance 2,694 2,884
Related Allowances      
Impaired loans with no related allowance recorded [Member] | Secured by non-farm, non-residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 19,528 24,411
Unpaid Principal Balance 19,528 24,411
Related Allowances      
Impaired loans with no related allowance recorded [Member] | Commercial and industrial [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance    100
Unpaid Principal Balance    100
Related Allowances      
Impaired loans with an allowance recorded [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 31,710 27,330
Unpaid Principal Balance 31,710 27,330
Related Allowances 11,609 11,084
Impaired loans with an allowance recorded [Member] | Construction, land development and other land loans [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 13,405 12,669
Unpaid Principal Balance 13,405 12,669
Related Allowances 7,278 6,800
Impaired loans with an allowance recorded [Member] | Secured by multi-family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 5,964  
Unpaid Principal Balance 5,964  
Related Allowances 1,553  
Impaired loans with an allowance recorded [Member] | Secured by non-farm, non-residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 11,304 13,159
Unpaid Principal Balance 11,304 13,159
Related Allowances 2,018 3,775
Impaired loans with an allowance recorded [Member] | Commercial and industrial [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 1,037 1,502
Unpaid Principal Balance 1,037 1,502
Related Allowances 760 509
Impaired loans [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 58,007 61,916
Unpaid Principal Balance 58,007 61,916
Related Allowances 11,609 11,084
Impaired loans [Member] | Construction, land development and other land loans [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 17,272 19,674
Unpaid Principal Balance 17,272 19,674
Related Allowances 7,278 6,800
Impaired loans [Member] | Secured by 1-4 family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 208 186
Unpaid Principal Balance 208 186
Impaired loans [Member] | Secured by multi-family residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 8,658 2,884
Unpaid Principal Balance 8,658 2,884
Related Allowances 1,553  
Impaired loans [Member] | Secured by non-farm, non-residential properties [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 30,832 37,570
Unpaid Principal Balance 30,832 37,570
Related Allowances 2,018 3,775
Impaired loans [Member] | Commercial and industrial [Member]
   
Schedule of carrying amount of impaired loans    
Ending balance 1,037 1,602
Unpaid Principal Balance 1,037 1,602
Related Allowances $ 760 $ 509