A cost is a decrease in an asset account or an increase in a liability account. (This is a generalization, but it’s pretty good for our purposes; characterizing costs fully is surprisingly difficult.)
An expense is a negative amount that appears on the income statement.
For example:
If you buy a box of pencils for cash, that’s a cost (decrease in cash); if you deduct that cost on your income statement, it’s also an expense.
If you buy a machine by making a down payment and taking out a loan for the balance of the price, that’s a cost (decrease in cash, increase in a liability). You generally won’t show the entire cost on your income statement the first year, but you’ll show depreciation; the depreciation is an expense.
If you trade inventory for a new truck, that’s a cost (decrease in inventory); the expense will again likely be depreciation over a number of years.