Colloidal metal nanocrystals (NCs) are known for their size- and shape-dependent localized surface plasmon resonances. Here, we explore the optical properties of Au NC assemblies as we control their number, arrangement, and inter-particle distance and exploit their properties in the design of optical metamolecules and metamaterials. We use lithography to define size- and shape-engineered topographical templates for the directed assembly of shape-tailored NCs into open and closed-packed oligomers. We map the energy, amplitude, and polarization dependence of their optical response showing examples of tailored Fano resonances and extrinsic chiroptical responses. We also use chemical exchange of the long ligands used in NC synthesis with more compact ligand chemistries to bring neighboring NCs into proximity and tailor a dielectric-to-metal phase transition in NC assemblies at DC and optical frequencies. We harness the solution-processability and physical properties of colloidal NCs to imprint and trigger the folding of NC superstructures to achieve large-area, active metamaterials, demonstrating broadband and reconfigurable 3D chiral optical metamaterials.