Segment Information (Details) (USD $)
In Thousands, unless otherwise specified
 3 Months Ended 6 Months Ended 12 Months Ended
Jun. 30, 2012
Jun. 30, 2011
Jun. 30, 2012
Jun. 30, 2011
Dec. 31, 2011
Segment Reporting Information [Line Items]          
Net interest income (loss) $ 6,222 $ 7,191 $ 12,265 $ 14,441  
Other revenue 3,812 2,684 6,864 4,808  
Other expense 8,002 8,847 15,580 16,892  
Depreciation 336 467 766 946  
Provision for loan losses 1,375 3,250 2,725 7,500  
Other intangibles 238 263 475 539  
Net allocations            
Income taxes benefit    (528)    (746)  
Segment profit (loss) 83 (2,424) (417) (5,882) 10,600
Segment assets 926,574 1,022,256 926,574 1,022,256  
Retail Segment [Member]
          
Segment Reporting Information [Line Items]          
Net interest income (loss) 1,821 1,961 3,699 4,004  
Other revenue 2,002 1,991 3,990 3,897  
Other expense 2,494 2,630 5,013 5,353  
Depreciation 236 273 473 548  
Provision for loan losses 627 460 859 845  
Other intangibles 238 263 475 539  
Net allocations 970 1,341 2,154 2,776  
Income taxes benefit    (15)    (27)  
Segment profit (loss) (742) (1,000) (1,285) (2,133)  
Segment assets 152,644 182,782 152,644 182,782  
Commercial Segment [Member]
          
Segment Reporting Information [Line Items]          
Net interest income (loss) 4,820 5,837 9,598 11,817  
Other revenue 736 134 1,435 452  
Other expense 1,468 1,598 2,559 2,426  
Depreciation         1  
Provision for loan losses 748 2,790 1,866 6,655  
Other intangibles            
Net allocations 2,545 2,888 4,970 5,607  
Income taxes benefit    (436)    (584)  
Segment profit (loss) 795 (869) 1,638 (1,836)  
Segment assets 468,471 537,493 468,471 537,493  
Treasury Segment [Member]
          
Segment Reporting Information [Line Items]          
Net interest income (loss) (377) (495) (867) (1,126)  
Other revenue 698 272 714 (120)  
Other expense 43 43 88 87  
Depreciation            
Provision for loan losses            
Other intangibles            
Net allocations 248 366 529 715  
Income taxes benefit    (77)    (135)  
Segment profit (loss) 30 (555) (770) (1,913)  
Segment assets 263,106 249,781 263,106 249,781  
Other Operations [Member]
          
Segment Reporting Information [Line Items]          
Net interest income (loss) (42) (112) (165) (254)  
Other revenue 376 287 725 579  
Other expense 3,997 4,576 7,920 9,026  
Depreciation 100 194 293 397  
Provision for loan losses            
Other intangibles            
Net allocations (3,763) (4,595) (7,653) (9,098)  
Income taxes benefit            
Segment profit (loss)            
Segment assets $ 42,353 $ 52,200 $ 42,353 $ 52,200