A couple of points to make here...first of all, X-Wings do not carry 4 Proton torpedoes each. Each X-Wing carriers 6 Proton Torpedoes loaded in 2 tubes, 3 in each tube. They can be fired individually, or in pairs. Star Wars squadrons are organized in groups of 12, Three flights of Four each, groups are 36 (Baron Soontir Fel is giving command of the 181st TIE Group, which is 3 Squadrons). Wings are 72. Star Destroyers carry between 60-72 TIE Fighters (Sometimes intersperesed with Assault Gunboats or Missile Boats).
The figure of 144 TIE Fighters comes from the original X-Wing game. On most of the X-Wing missions if you attacked an Imperial Star Destroyer (or even a Nebulon-B Frigate) the ship would launch waves of TIE Fighters up to 144 at you. Or TIE Interceptors, or some combination there of. (This is noted repeatedly by the game programmers in the Strategy Guide).
Regarding the Shield Generators...well if you played the games or read any of the EU Material, it's all pretty consistant: The big domes on top of the Star Destroyer tower are Shield Generators, with the Main Communications and Sensor array Sandwhiched down between them. Incidently enough, the main Bridge is actually located around the Middle of the Front of that tower. If you don't want to move into the EU Material, then I direct your attention to the ROTJ Film. Shortly after a pair of X-Wing hit a "tower bubble" (we'll call them that to avoid ticking anyone off) the Imperial Officer declares that they've lost the Bridge Deflector shield.
I'm going to get all technical now. The Standard ISD is somewhere between 1 Kilometer and 1.5 Kilometers long. It has a pair of shield generators (this is confirmed even if they aren't the dome "tower bubbles"). The Executor is between 6 and 18 Kilometers long (It's given various lengths by differant sources). This means it would require many, many more shield generators to protect itself, and obviously, these shield generators would serve differant purposes (Aft, fore, port, starboard, BRIDGE, etc etc). From various places and differant shots we can see the Executor with up to 9 "tower bubbles" on various portions of it's hull. Some of which make absolutly no sense as sensor bubbles...but would make perfect sense as Shield Generators.
So from a strictly movie persepctive: IF tower bubble blowing = shield failure, THEN tower bubble = shield generator. Oops, I'm using logic on the internet, aren't I?
From the EU Perspective, it's confirmed in a number of differant sources, but Before the Storm is most noticeable when it describes Executor-class Star Destroyers as having "shield generators at midline". Which meshes nicely with the shield generators smattered around the hull of the Executor in the movie. Or I guess I should call them tower bubbles.
Whew. That debate out of the way, let's talk about the X-Wing VS Star Destroyer debate. Star Destroyers are armed with Ion Cannons and Turbolasers. Yup, Turbolasers. No Quadlaser cannons for these bad boys. What are Turbolasers? Turbolasers are like the large cannons on a battleship. Now, imagine, just for a moment, using a Battleship cannon to blast say, a P-51 out of the sky. Going to have any luck with that? Probably not. That's what a Star Destroyer is armed with. One hit will kill anyone with that kind of power, but you have to hit your target, and that's not easy. So, fast forward a bit.
Once the X-Wings have closed with the Star Destroyer, hitting an X-Wing with the weaponary is pretty much impossible. This where the primary tactic of the Rebel Fighters comes in handy, TRD, Trench Run Defense/Disease (depending on the source). Star Destroyers have natural overhanging channels on their hull. I always took this to be the armor overhanging the inner "soft" workings of the ship. Presumably, operating this close to the hull puts you inside the protection radius of the ISD's shields, and if you're underneath the armor...well you do the math.
All that being said, most Star Destroyers are taking out by hitting the bridge, or the Reactor (which is the large dome beneath the hull of the ship). While hitting the reactor certainly makes sense, due to the development of auxilary bridges, the bridge is a less viable option. However, we only see a Victory-class destroyed by a hit to the bridge, which is a much early designed Star Destroyer and possibly lacks the devleopment of an auxilary bridge.
It's not very difficult to take out the shield generator, either. As it's a fairly exposed target. In the games it's not possible to sit still and shoot it again and again, but multiple passes seem to work better anyway, as you avoide most of the return fire this way. And once the shield generators are gone calling in a group of bombers is well within the realm of possability.
Almost done, just a few more things to say. Most strikes against Star Destroyers are supported by either B-Wings or Y-Wings. Obviously, B-Wings, with 3 Ion Cannons and 3 Laser Cannons, and 12 proton torpedoes in two tubes of six, are the better choice over Y-Wings, which have 2 Ion Cannons, 2 Laser Cannons, and 8 proton torpedoes, 4 in each tube, are much better suited for the task. Not to mention the speed of B-Wings (96 MGLT at normal ELS Settings) compared to Y-Wings (50 MGLT at normal ELS Settings). In fact, a Y-Wing can only obtain the speed of a TIE Fighter by putting all energy into engines from it's weapons and it's shields. And even then, if the TIE Fighter does the same it would vastly outstrip the Y-Wing.
There's also a variety of differant weapons system in use for the Alliance. Including Proton Torpedoes, Advanced Proton Tropedoes, Heavy Rockets (Proton Torpedoes that are slower but with a larger warhead), and Space Bombs. They are useful in differant sets of circumstances, for example, Space Bombs can take up to 30 minutes to reach their target (they are painfully slow) and are large and easily targetable, but do massive amounts of damage.
Aside from the more traditional warhead damage, when supported by B-Wings or Y-Wings on a bombing operation a new weapon becomes useful: the Ion Cannon. It's been listed here as an anti-shield weapon, and this is very true, but the Ion Cannon also has the ability to disable electronic systems. I find that when bombing an ISD in either a B-Wing or a Y-Wing, knocking out it's shields and then disabling it allows it die much much quicker then just shooting it with lasers. Also, XVT And XWA see the addition of Ion Pulse warheads, which do the job of an Ion Cannon on a much larger scale.
Also, in the Games and Novels, typical attacks on Star Destroyers are rarely carried out by just a lone group of X-Wings. And when they are (X-Wing and XvT Come to mind) they are usually in the middle of a re-fueling or re-arming process, when the combat status of the ship is somewhat less then full power.
In fact, in at least one case in the EU (And I'm fairly certain it happened more then once) a Rebel pilot flies her ship inside of an ISD And begins to damage critical sub systems from inside of the ship, resulting in massive computer failure on board the already battle damaged ship.
So in conclusion, when all factors (Shield Generators, Weapons Systems, Fighter capabilites) are taken into account, it's quite reasonable that an X-Wing squadron could destroy an Imperial Star Destroyer, probably in the time frame of between 15 and 20 minutes.
EDIT: As for a fighter that is ideal for taking out an Imperial Star Destroyer...take a look at the E-Wing. This ship is as manuverable as the TIE Advanced and features a speed rating (135 MGLT) equal to an A-Wing. It also carries 3 Laser Cannons for dogfighting and no less then 16 proton Torpedoes in two tubes of eight each. Talk about heavy fire power. Usually it's coupled with the K-Wing Bomber, which in the NR Era carries an "Egg" (if it has another official name I don't know what it is) that does massive damaging, detonating on the edge of a shield boundary in an effort to overload the shield generator (and an effort that is usually succesful). When hitting one or two of these against a Star Destroyer they are designed specifically to "break the back" of the Star Destroyer. That ship features a centrle single Ion Cannon, four Laser cannons, a speed fo 96 MGLT, an Egg, a load of Heavy Rockets, a load of Concussion missiles, and a crew of two. Some variants have a crew of three with a Quad Laser turret mounted underneath the forward cockpit, but the thing handles like a dump truck. Still, it can engage and range and with heavy striking power. IT's a very nice ship. And when coupled with E-Wing escorts (change the Torpedoes to Advanced Missiles, to make them extra good against fighters) which can absorb a tremendous amount of hull damage (assuming it's shields fail) and you've got a killer combination for taking out Capital ships at a much smaller man power cost.