Dialog instance and Application Server are generally used synonymously.
A SAP instance is basically an administrative unit providing one or more services. Check the below summary:
ABAP AS
-------
Dialog Instance = Instance where dispatcher run with other work processes like dialog, background, spool etc
Central Instance for AS ABAP = Message Server + Enqueue work process
Central Services Instance for AS ABAP (ASCS) = Message Server + Enqueue Server (when high availability is used)
http://help.sap.com/saphelp_nw04s/helpdata/EN/43/e6b0a0d7633886e10000000a1553ed/content.htm
Central System = Database + Central Instance
Java
----
Dialog Instance = Java dispatcher (without SDM)
Central Instance for AS JAVA = SDM + Dialog Instance (Where SDM runs)
Central Services Instance for AS JAVA (SCS) = Message Server + Enqueue Service
Central System = Database + Central Instance + Central Services Instance
ASCS standard for ABAP SAP Central Services.
SCS standards for SAP Central services.
You can read following also.
http://help.sap.com/saphelp_nwpi711/helpdata/en/47/fe7aa040e23c8be10000000a42189c/content.htm